Apache Spark: how to do a distinct count and count together?
<p>I would like to do a distinct count and a count at the same time.
Let me explain you through a simple example.
Here is my input data:</p>

<pre><code>arbre;pommier
fruit;pomme
fruit;pomme
fruit;fraise
fruit;ichigo
arbre;cerisier
arbre;abricotier
sport;foot
sport;rugby
arbre;abricotier
arbre;abricotier
arbre;bananier
fruit;ananas
sport;basket
arbre;abricotier
arbre;abricotier
</code></pre>

<p>As an output, I want to get that:</p>

<pre><code>//type;count;distinct-count
arbre;9;3
fruit;5;4
sport;3;3
</code></pre>

<p>With the word count example, I can compute the number of time a word appears (so here is count). And with the function <code>distinct().groupeByKey()</code>, I manage to compute the distinct count (here is distinct-count).</p>

<p>However, I can't figure out how to do that on the same RDD and not two separate RDDs.</p>

<p>If you have an idea, documentation or a link about this subject I would be very thankful.</p>