Filtering data in an RDD
<p>I have managed to pre process my data in pyspark to get something like this</p>

<pre><code>[(u'key1', u'1'), (u'key2', u'1'), (u'key1', u'2'), (u'key3', u'2'), (u'key4', u'1'), (u'key1', u'4'), (u'key5', u'1'), (u'key6', u'2'), (u'key7', u'4'), (u'key8', u'5'), (u'key9', u'6'), (u'key10', u'7')]
</code></pre>

<p>Now I need to filter based on these conditions :</p>

<p>1) filter values associated to atleast 2 keys.</p>

<p>output - only those (k,v) pairs which has '1','2','4' as values should be present since they are associated with more than 2 keys</p>

<pre><code> [(u'key1', u'1'), (u'key2', u'1'), (u'key1', u'2'), (u'key3', u'2'), (u'key4', u'1'), (u'key1', u'4'), (u'key5', u'1'), (u'key6', u'2'), (u'key2', u'4')]
</code></pre>

<p>2) filter keys which are associated to atleast 2 values</p>

<p>output - only those (k,v) pairs which has key1, key2 as keys should be there since they are associated with atleast 2 values</p>

<pre><code>[(u'key1', u'1'), (u'key2', u'1'), (u'key1', u'2'), (u'key1', u'4'), (u'key2', u'4')]
</code></pre>

<p>Any suggestions would be of great help.</p>

<p>Update : I used groupBy and a filter to group for keys with mutiple values</p>

<pre><code> [(u'key1', [u'1', u'2', u'4']), (u'key2',[u'1', u'4'])]
</code></pre>

<p>Now how do I split this (key, list(values)) to individual (k,v) pair to apply further transformation ?</p>