Pyspark: multiplying columns from different tables
<p>I have these two dataframes:</p>

<pre><code>df1 = sc.parallelize([
['u1', 0.5],
['u2', 0.2],
['u3', 0.1],
['u4', 0.9],
['u5', 0.7]
]).toDF(('person', 'score'))

df2 = sc.parallelize([
['d1', 0.0],
['d2', 0.5],
['d3', 0.7],
]).toDF(('dog', 'score'))
</code></pre>

<p>What I need to do is creating another dataframe whose schema would be </p>

<blockquote>
  <p>person, dog, score_person * score_dog</p>
</blockquote>

<p>so basically multiplying the column <code>score</code> in both dataframes and keeping the two first columns. This multiplication has to take place for each possible couple of factors, i.e. each person with each dog, so that my result dataframe would have 15 rows.</p>

<p>I can't find a way to obtain this, it seems to me that it has to pass through a SELECT on both dataframes but no JOIN nor UNION can help.</p>