Comparing columns in Pyspark
<p>I am working on a PySpark DataFrame with n columns. I have a set of m columns (m &lt; n)  and my task is choose the column with max values in it.</p>
<p>For example:</p>
<p>Input: PySpark DataFrame containing :</p>
<p><code>col_1 = [1,2,3], col_2 = [2,1,4], col_3 = [3,2,5]</code></p>
<p>Ouput :</p>
<p><code>col_4 = max(col1, col_2, col_3) = [3,2,5]</code></p>
<p>There is something similar in pandas as explained in <a href="https://stackoverflow.com/questions/27474921/compare-two-columns-using-pandas">this</a> question.</p>
<p>Is there any way of doing this in PySpark or should I change convert my PySpark df to Pandas df and then perform the operations?</p>