spark (Scala) dataframe filtering (FIR)
<p>Let say I have a dataframe ( stored in scala val as <code>df</code>) which contains the data from a csv:</p>

<pre><code>time,temperature
0,65
1,67
2,62
3,59
</code></pre>

<p>which I have no problem reading this from file as a spark dataframe in scala language.</p>

<p>I would like to add a filtered column (by filter I meant signal processing moving average filtering), (say I want to do <code>(T[n]+T[n-1])/2.0</code>):</p>

<pre><code>time,temperature,temperatureAvg
0,65,(65+0)/2.0
1,67,(67+65)/2.0
2,62,(62+67)/2.0
3,59,(59+62)/2.0
</code></pre>

<p>(Actually, say for the first row, I want <code>32.5</code> instead of <code>(65+0)/2.0</code>. I wrote it to clarify the expected 2-time-step filtering operation output)</p>

<p>So how to achieve this? I am not familiar with spark dataframe operation which combine rows iteratively along column...</p>