PySpark: how to resample frequencies
<p>Imagine a Spark Dataframe consisting of value observations from variables. Each observation has a specific timestamp and those timestamps are not the same between different variables. This is because the timestamp is generated when the value of a variable changed and is recorded. </p>

<pre><code>#Variable     Time                Value
#852-YF-007   2016-05-10 00:00:00 0
#852-YF-007   2016-05-09 23:59:00 0
#852-YF-007   2016-05-09 23:58:00 0
</code></pre>

<p><strong>Problem</strong> I would like to put all variables into the same frequency (for instance 10min) using forward-fill. To visualize this, I copied a page from the Book "Python for Data Analysis". <strong>Question:</strong> How to do that on a Spark Dataframe in an <strong>efficient</strong> way?</p>

<p><a href="https://i.stack.imgur.com/K5Fxr.png" rel="noreferrer"><img src="https://i.stack.imgur.com/K5Fxr.png" alt="Python for Data Analysis"></a></p>