Convert rows of pandas Dataframe into an iterable list of strings
<p>Imagine I have a dataframe, df, that has 2 columns, a USER_ID and a PRODUCT they bought.  </p>

<pre><code>df
USER_ID     |     PRODUCT
1                 a
1                 b
1                 c
2                 d
2                 a
2                 k
</code></pre>

<p>I want to convert this DataFrame into a new one, df2, where each row is one user and the products are aggregated into a list of string.  </p>

<pre><code>df2
USER_ID     |     PRODUCT
1                 [a,b,c]
2                 [d,a,k]
</code></pre>

<p>Finally, I’d like to be able to find the intersection between two user’s PRODUCT lists.</p>

<p>I’m able to create the second dataframe, but the method I’m using results in a list that is somehow not iterable.</p>

<p>Specifically i do:  
    df2 = df1.groupby(‘USER_ID)[‘PRODUCT’].agg(lambda x: x.tolist())</p>

<p>which gives me a series that I convert back to a dataframe.</p>

<pre><code>df2 = df2.to_frame()
</code></pre>

<p>which gives me the df2 I’m looking for but each list of products has length = 1, thus I can’t compare 1 to another to find the intersection of products.   For instance, when I execute:</p>

<pre><code>s1 = df2.PRODUCT[df2.USER_ID == 1] 
s2 = df2.PRODUCT[df2.USER_ID == 2]

common_elements = list(set(s1).intersection(set(s2)))
common_elements
</code></pre>

<p>The result is an empty list rather than [a].  What am I doing wrong? </p>