Count the number of sql dataframe in spark by map function in pyspark
<p>A newbie in spark and have a problem about <code>map</code> function on data frame. I have a spark sql dataframe, named <code>df</code>, assuming it is like:</p>

<pre><code>+----------+------------+------+
|      time|         tag| value|
+----------+------------+------+
|1399766400|A00000000001|1000.0|
|1399766401|A00000000002|1001.0|
+----------+------------+------+
</code></pre>

<p>I can select part of them based on the tag value with the command:</p>

<pre><code>temp = sqlContext.sql("SELECT * FROM df WHERE tag = 'A00000000001'")
temp.show(1)
</code></pre>

<p>then we have:</p>

<pre><code>+----------+------------+------+
|      time|         tag| value|
+----------+------------+------+
|1399766400|A00000000001|1000.0|
+----------+------------+------+
</code></pre>

<p>Currently, I have a list </p>

<pre><code>x = ["SELECT * FROM df WHERE tag = 'A00000000001'", "SELECT * FROM df WHERE tag = 'A00000000002'"]
</code></pre>

<p>which has been stored as RDD variable and I would like to apply <code>map</code> function on it to count the number of dataframe selected based on them, I tried the function like:</p>

<pre><code>y = x.map(lambda x: sqlContext.sql(x).count())
y.take(2)
</code></pre>

<p>I supposed that the return value should be <code>[1, 1]</code>, but it gives the error:</p>

<pre><code>TypeError: 'JavaPackage' object is not callable
</code></pre>

<p>Is it possible to execute a map function on a dataframe with this method? if not, how should I do.</p>