PySpark: How to group a column as a list when joining two spark dataframes?
<p>I want to join the following spark dataframes on Name:</p>

<pre><code>df1 = spark.createDataFrame([("Mark", 68), ("John", 59), ("Mary", 49)], ['Name', 'Weight'])

df2 = spark.createDataFrame([(31, "Mark"), (32, "Mark"), (41, "John"), (42, "John"), (43, "John")],[ 'Age', 'Name'])
</code></pre>

<p>but I want the result to be the following dataframe:</p>

<pre><code>df3 = spark.createDataFrame([([31, 32], "Mark", 68), ([41, 42, 43], "John", 59), `(None, "Mary", 49)],[ 'Age', 'Name', 'Weight'])
</code></pre>