TypeError: object of type 'PipelinedRDD' has no len()
<p>everyone! I meet this error log when running a python code in spark </p>

<p>code :</p>

<p><strong>main</strong>.py</p>

<pre><code>sc = SparkContext(appName="newsCluster")
sc.addPyFile("/home/warrior/gitDir/pysparkCode/clusterNews/wordSeg.py")
sc.addPyFile("/home/warrior/gitDir/pysparkCode/clusterNews/sparkClusterAlgorithm.py")
wordseg = wordSeg()
clustermanage = sparkClusterAlgorithm()
files = sc.wholeTextFiles("hdfs://warrior:9000/testData/skynews")
file_list = files.map(lambda item: item[1])
file_wc_dict_list = file_list.map(lambda file_content:wordseg.GetWordCountTable(file_content))
file_wc_dict_list.persist()
all_word_dict = wordseg.updateAllWordList(file_wc_dict_list)
</code></pre>

<p>wordSeg.py</p>

<pre><code>def updateAllWordList(self, newWordCountDictList):
    '''
        description: input an new file then update the all word list
        input:
            newWordCountDict: new input string word count dict
        output:
            all_word_dict
    '''
    n = len(newWordCountDictList)
    all_word_list = []
    all_word_dict = {}
    for i in range(0,n):
        all_word_list = list(set(all_word_list + newWordCountDictList[i].keys()))
    for i in range(0,len(all_word_list)):
        all_word_dict[all_word_list[i]]=0
    return all_word_dict
</code></pre>

<p>.......
.......</p>

<p>when spark-submit <strong>main</strong>.py
output error log:</p>

<pre><code>Traceback (most recent call last):                                              
File "/home/warrior/gitDir/pysparkCode/clusterNews/__main__.py", line 31, in &lt;module&gt;
  all_word_dict =     wordseg.updateAllWordList(file_wc_dict_list)#file_wc_dict_list.map(lambda     file_wc_dict:wordseg.updateAllWordList(file_wc_dict))
File "/home/warrior/gitDir/pysparkCode/clusterNews/wordSeg.py", line 54, in updateAllWordList
  n = len(newWordCountDictList)
TypeError: object of type 'PipelinedRDD' has no len()
</code></pre>

<p>how to solve it!!
thanks!!</p>