Spark 2.0 groupBy column and then get max(date) on a datetype column
<p>I'm using Spark 2.0 in Java.
I have a Dataset which look like this:</p>

<pre><code>------+----+----+----+----------------+
    ID|col1|col2|col3|            date|
------+----+----+----+----------------+
981200| a  | q  | z  |      2010-12-30|
968800| s  | w  | x  |      2010-12-31|
966500| d  | e  | c  |      2010-11-02|
966500| f  | r  | v  |      2010-12-01|
981200| g  | t  | b  |      2010-12-03|
</code></pre>

<p>I wanto to groupBy ID and get only the row with the "last" date (the most recent one). The type of column "date" is <strong>date</strong>.</p>

<p>In my case the result would be</p>

<pre><code>------+----------------+
    ID|       max(date)|
------+----------------+
981200|      2010-12-30|
968800|      2010-12-31|
966500|      2010-12-01|
</code></pre>

<p>I tried</p>

<pre><code>    Dataset&lt;Row&gt; df = old_df.groupBy("ID").max("date");
</code></pre>

<p>but fails with error</p>

<blockquote>
  <p>Exception in thread "main" org.apache.spark.sql.AnalysisException: 
      "date" is not a numeric column.
      Aggregation function can only be applied on a numeric column.;</p>
</blockquote>

<p>Is there a way to get the max date (using type date) in Spark?</p>

<p>Thanks in advance.</p>