Spark Dataframe: How to aggregate both numerical and nominal columns
<p>I am using Spark Dataframes and have dataframe <code>df</code> similar to this:</p>

<pre><code>id: String | amount: Double | donor: String
--------------------------------------------
1          | 50             | Mary
2          |100             | Michael
1          | 60             | Minnie
1          | 20             | Mark
2          | 55             | Mony
</code></pre>

<p>I want to aggregate my dataframe in one go and get this output:</p>

<pre><code>id: String | amount: Double | donor: Seq[String]
--------------------------------------------
1          |130             | {Mary,Minnie,Mark}
2          |155             | {Michael, Mony}
</code></pre>

<p>So I want to do something like:</p>

<p><code>df.groupyBy("id").agg(sum("amount"),_?Seq?_("donor"))</code></p>

<p>Aggregating the sum of the numbers is easy, but I can't find a way to aggregate the text content as a Sequence or Array (or any similar type that is Iterable). How can I do this in scala/spark?</p>

<p>EDIT:</p>

<p>I am looking for some spark Dataframe or RDD based function to do the collection of strings. Functions as the below mentioned <code>collect_set</code> are Hive based and I need specific dependencies for that. But I am not using Hive at all in my project.</p>