Spark program to read XML and parse it using databricks spark_xml jar
<p>Currently I am trying to load an xml file and parse it using databricks spark_xml.</p>

<pre><code>def loadXMLData(sqlContext: SQLContext) = {
var df : DataFrame = null
var newDf : DataFrame = null
import sqlContext.implicits._
df =sqlContext.read.format("com.databricks.spark.xml").option("rowTag","book").load("books_new.xml")
val newRdd: RDD[Row] = df.rdd
//printing each Row
newRdd.collect().foreach(a=&gt;println(a))
df.printSchema()
df.registerTempTable("book")
sqlContext.sql("""select * from book """).show()

}
</code></pre>

<p>While iterating through Row i have a column email and I want to find out the id which doesnot have @ and update them as null before loading it into hive.
Also after registering as temp table I want to load the data in hive for which I added : </p>

<pre><code>libraryDependencies += "org.apache.spark" %% "spark-hive" % "1.5.0"
</code></pre>

<p>but it says cannot resolve symbol hive when I give :</p>

<pre><code>val sqlContext = new org.apache.spark.sql.hive.HiveContext(sc)
</code></pre>

<p>how to go about this</p>