How to know which is the RDD type inferred by Spark using Scala
<p>I was trying the follow example</p>

<pre><code>val lista = List(("a", 3), ("a", 1), ("b", 7), ("a", 5))
val rdd = sc.parallelize(lista)
</code></pre>

<p>Then in the shell I get the following</p>

<pre><code>rdd: org.apache.spark.rdd.RDD[(String, Int)] = ParallelCollectionRDD[40] at parallelize at &lt;console&gt;:30
</code></pre>

<p>But for some reason I still haven't figured out I was able to execute this sentence</p>

<pre><code>val resAgg = rdd.aggregateByKey(new HashSet[Int])(_+_, _++_)
</code></pre>

<p>Getting this in the shell</p>

<pre><code>resAgg: org.apache.spark.rdd.RDD[(String, scala.collection.mutable.HashSet[Int])] = ShuffledRDD[41] at aggregateByKey at &lt;console&gt;:32
</code></pre>

<p>So I have some questions:</p>

<p>1.- What is the real RDD type of the var named rdd? because in the shell it shows is of the type org.apache.spark.rdd.RDD[(String, Int)] but looking on the API the RDD class does not have a method aggregateByKey. By the way JavaPairRDD class does it have the aggregateByKey method</p>

<p>2.- How Can I verify/know the real type of a RDD</p>

<p>3.- What is that ParallelCollectionRDD showed up? I looked for it on github and I found is a private class so I guess is that the reason why is does not appears on the scala API, but what it is for?</p>

<p>I was using Spark 1.6.2</p>