Extracting `Seq[(String,String,String)]` from spark DataFrame
<p>I have a spark DF with rows of <code>Seq[(String, String, String)]</code>. I'm trying to do some kind of a <code>flatMap</code> with that but anything I do try ends up throwing</p>

<blockquote>
  <p>java.lang.ClassCastException: org.apache.spark.sql.catalyst.expressions.GenericRowWithSchema cannot be cast to scala.Tuple3</p>
</blockquote>

<p>I can take a single row or multiple rows from the DF just fine</p>

<pre><code>df.map{ r =&gt; r.getSeq[Feature](1)}.first
</code></pre>

<p>returns </p>

<pre><code>Seq[(String, String, String)] = WrappedArray([ancient,jj,o], [olympia_greece,nn,location] .....
</code></pre>

<p>and the data type of the RDD seems correct. </p>

<p><code>org.apache.spark.rdd.RDD[Seq[(String, String, String)]]</code></p>

<p>The schema of the df is</p>

<pre><code>root
 |-- article_id: long (nullable = true)
 |-- content_processed: array (nullable = true)
 |    |-- element: struct (containsNull = true)
 |    |    |-- lemma: string (nullable = true)
 |    |    |-- pos_tag: string (nullable = true)
 |    |    |-- ne_tag: string (nullable = true)
</code></pre>

<p>I know this problem is related to spark sql treating the RDD rows as <code>org.apache.spark.sql.Row</code> even though they idiotically say that it's a <code>Seq[(String, String, String)]</code>. There's a related question (link below) but the answer to that question doesn't work for me. I am also not familiar enough with spark to figure out how to turn it into a working solution.</p>

<p>Are the rows <code>Row[Seq[(String, String, String)]]</code> or <code>Row[(String, String, String)]</code> or <code>Seq[Row[(String, String, String)]]</code> or something even crazier.</p>

<p>I'm trying to do something like</p>

<pre><code>df.map{ r =&gt; r.getSeq[Feature](1)}.map(_(1)._1)
</code></pre>

<p>which appears to work but doesn't actually</p>

<pre><code>df.map{ r =&gt; r.getSeq[Feature](1)}.map(_(1)._1).first
</code></pre>

<p>throws the above error. So how am I supposed to (for instance) get the first element of the second tuple on each row?</p>

<p>Also <strong>WHY</strong> has spark been designed to do this, it seems idiotic to claim that something is of one type when in fact it isn't and can not be converted to the claimed type.</p>

<hr>

<p>Related question: <a href="https://stackoverflow.com/questions/32727518/genericrowwithschema-exception-in-casting-arraybuffer-to-hashset-in-dataframe-to">GenericRowWithSchema exception in casting ArrayBuffer to HashSet in DataFrame to RDD from Hive table</a></p>

<p>Related bug report: <a href="http://search-hadoop.com/m/q3RTt2bvwy19Dxuq1&amp;subj=ClassCastException+when+extracting+and+collecting+DF+array+column+type" rel="noreferrer">http://search-hadoop.com/m/q3RTt2bvwy19Dxuq1&amp;subj=ClassCastException+when+extracting+and+collecting+DF+array+column+type</a></p>