spark sql window function lag
<p>I am looking at the <code>window</code> slide function for a Spark <code>DataFrame</code> in Scala.</p>
<p>I have a <code>DataFrame</code> with columns <code>Col1</code>, <code>Col2</code>, <code>Col3</code>, <code>date</code>, <code>volume</code> and <code>new_col</code>.</p>
<pre><code>Col1    Col2    Col3    date     volume new_col
                        201601  100.5   
                        201602  120.6   100.5
                        201603  450.2   120.6
                        201604  200.7   450.2
                        201605  121.4   200.7`
</code></pre>
<p>Now I want to add a new column with name(<code>new_col</code>) with one row slided down, as shown above.</p>
<p>I tried below option to use the window function.</p>
<pre><code>val windSldBrdrxNrx_df = df.withColumn(&quot;Prev_brand_rx&quot;, lag(&quot;Prev_brand_rx&quot;,1))
</code></pre>
<p>Do you have any suggestion ?</p>