How to convert a case-class-based RDD into a DataFrame?
<p>The <a href="http://spark.apache.org/docs/latest/sql-programming-guide.html#inferring-the-schema-using-reflection" rel="noreferrer">Spark documentation</a> shows how to create a DataFrame from an RDD, using Scala case classes to infer a schema. I am trying to reproduce this concept using <code>sqlContext.createDataFrame(RDD, CaseClass)</code>, but my DataFrame ends up empty. Here's my Scala code:</p>

<pre><code>// sc is the SparkContext, while sqlContext is the SQLContext.

// Define the case class and raw data
case class Dog(name: String)
val data = Array(
    Dog("Rex"),
    Dog("Fido")
)

// Create an RDD from the raw data
val dogRDD = sc.parallelize(data)

// Print the RDD for debugging (this works, shows 2 dogs)
dogRDD.collect().foreach(println)

// Create a DataFrame from the RDD
val dogDF = sqlContext.createDataFrame(dogRDD, classOf[Dog])

// Print the DataFrame for debugging (this fails, shows 0 dogs)
dogDF.show()
</code></pre>

<p>The output I'm seeing is:</p>

<pre><code>Dog(Rex)
Dog(Fido)
++
||
++
||
||
++
</code></pre>

<p>What am I missing?</p>

<p>Thanks!</p>