Unable to create dataframe from RDD of Row using case class
<p>Using Spark 2.x, it seems I cannot create a Dataframe using a RDD of Row composed of case classes.</p>

<p>It did work fine on Spark 1.6.x but fails on 2.x the following runtime exception:</p>

<pre><code>java.lang.RuntimeException: Timestamp is not a valid external type for schema of struct&lt;seconds:bigint,nanos:int&gt;
</code></pre>

<p>preceded by a bunch of generated code from Catalyst.</p>

<p>Here is the snippet (simplified version of what I am doing):</p>

<pre><code>package main

import org.apache.spark.sql.{SparkSession, Row}
import org.apache.spark.sql.types.{IntegerType, LongType, StructField, StructType}

object Test {

  case class Timestamp(seconds: Long, nanos: Int)

  val TIMESTAMP_TYPE = StructType(List(
    StructField("seconds", LongType, false),
    StructField("nanos", IntegerType, false)
  ))

  val SCHEMA = StructType(List(
    StructField("created_at", TIMESTAMP_TYPE, true)
  ))

  def main(args: Array[String]) {

    val spark = SparkSession.builder().getOrCreate()

    val rowRDD = spark.sparkContext.parallelize(Seq((0L, 0))).map {
      case (seconds: Long, nanos: Int) =&gt; {
        Row(Timestamp(seconds, nanos))
      }
    }

    spark.createDataFrame(rowRDD, SCHEMA).show(1)
  }
}
</code></pre>

<p>I am not sure if this is a Spark bug or something I missed in the documentation (I know Spark 2.x introduced runtime Row encoding validation, maybe this is related)</p>

<p>Help much appreciated</p>