Sum operation on PySpark DataFrame giving TypeError when type is fine
<p>I have such DataFrame in PySpark (this is the result of a take(3), the dataframe is very big):</p>

<pre><code>sc = SparkContext()
df = [Row(owner=u'u1', a_d=0.1), Row(owner=u'u2', a_d=0.0), Row(owner=u'u1', a_d=0.3)]
</code></pre>

<p>the same <em>owner</em> will have more rows. What I need to do is summing the values of the field a_d per owner, after grouping, as </p>

<pre><code>b = df.groupBy('owner').agg(sum('a_d').alias('a_d_sum'))
</code></pre>

<p>but this throws error</p>

<blockquote>
  <p>TypeError: unsupported operand type(s) for +: 'int' and 'str'</p>
</blockquote>

<p>However, the schema contains double values, not strings (this comes from a printSchema()):</p>

<pre><code>root
|-- owner: string (nullable = true)
|-- a_d: double (nullable = true)
</code></pre>

<p>So what is happening here?</p>