SparkSQL: Use custom column in GROUP BY
<p>I am using Spark 1.3.0 .In my application , I have created a dataframe with following <code>columns (col1, col2, col3,col4)</code></p>

<p>I have to derive a custom column(a new column) in the dataframe.
So my SQL query looks like</p>

<pre><code>SELECT (col1 + col1/10) as col1 , col2, col3 , col4 FROM table GROUP BY col1 , col2
</code></pre>

<p>My requirement in to group by new custom column and not the old col1. But the above query doesnt work properly. It still group by old col1. I know I have given the same name to new custom column as old column name. Even if try to give new name to custom column like:</p>

<pre><code>SELECT (col1 + col1/10) as new_col1 , col2, col3 , col4 FROM table GROUP BY new_col1 , col2
</code></pre>

<p>It fails with following error:</p>

<pre><code>org.apache.spark.sql.AnalysisException: cannot resolve 'new_col1' given input columns..
</code></pre>

<p>Any suggestion how to achieve this??
Thanks</p>