RDD[Array[String]] to Dataframe
<p>I am new to Spark and Hive and my goal is to load a delimited(lets say csv) to Hive table. After a bit of reading I found out that the path to load the data into Hive is <code>csv-&gt;dataframe-&gt;Hive</code>.(Please correct me if I am wrong). </p>

<pre><code>CSV:
1,Alex,70000,Columbus
2,Ryan,80000,New York
3,Johny,90000,Banglore
4,Cook, 65000,Glasgow
5,Starc, 70000,Aus
</code></pre>

<p>I read the csv file be using below command:</p>

<pre><code>val csv =sc.textFile("employee_data.txt").map(line =&gt; line.split(",").map(elem =&gt; elem.trim))
csv: org.apache.spark.rdd.RDD[Array[String]] = MapPartitionsRDD[29] at map at &lt;console&gt;:39
</code></pre>

<p>Now I am trying to convert this RDD to Dataframe and using below code:</p>

<pre><code>scala&gt; val df = csv.map { case Array(s0, s1, s2, s3) =&gt; employee(s0, s1, s2, s3) }.toDF()
df: org.apache.spark.sql.DataFrame = [eid: string, name: string, salary: string, destination: string]
</code></pre>

<p>employee is a case class and I am using it as a schema definition.</p>

<pre><code>case class employee(eid: String, name: String, salary: String, destination: String)
</code></pre>

<p>However when I do <code>df.show</code> I am getting below error:</p>

<blockquote>
  <p>org.apache.spark.SparkException: Job aborted due to stage failure:
  Task 0 in stage 10.0 failed 4 times, most recent failure: Lost task
  0.3 in stage 10.0 (TID 22, user.hostname): scala.MatchError: [Ljava.lang.String;@88ba3cb (of class
  [Ljava.lang.String;)</p>
</blockquote>

<p>I was expecting a dataframe as a output. I know why I might be getting this error because the values in RDD are stored in <code>Ljava.lang.String;@88ba3cb</code> format and I need to use <code>mkString</code> to get the actual values but I am not able to find how to do it. I appreciate your time.</p>