Spark: Merge 2 dataframes by adding row index/number on both dataframes
<p>Q: Is there is any way to merge two dataframes or copy a column of a dataframe to another in PySpark?</p>

<p>For example, I have two Dataframes:</p>

<pre><code>DF1              
C1                    C2                                                        
23397414             20875.7353   
5213970              20497.5582   
41323308             20935.7956   
123276113            18884.0477   
76456078             18389.9269 
</code></pre>

<p>the seconde dataframe</p>

<pre><code>DF2
C3                       C4
2008-02-04               262.00                 
2008-02-05               257.25                 
2008-02-06               262.75                 
2008-02-07               237.00                 
2008-02-08               231.00 
</code></pre>

<p>Then i want to add C3 of DF2 to DF1 like this:</p>

<pre><code>New DF              
    C1                    C2          C3                                              
    23397414             20875.7353   2008-02-04
    5213970              20497.5582   2008-02-05
    41323308             20935.7956   2008-02-06
    123276113            18884.0477   2008-02-07
    76456078             18389.9269   2008-02-08
</code></pre>

<p>I hope this example was clear.</p>