Reading JSON object with JSONArray inside to RDD using Scala and without using Dataframe
<p>I was able to read a json object to RDD using the following code without using dataframe, here is my JSON Object:</p>

<pre><code>{"first":"John","last":"Smith","address":{"line1":"1 main street","city":"San Francisco","state":"CA","zip":"94101"}}
</code></pre>

<p>Here is the code for reading it to RDD:</p>

<pre><code>package com.spnotes.spark
import com.fasterxml.jackson.annotation.JsonProperty
import com.fasterxml.jackson.core.JsonParseException
import com.fasterxml.jackson.databind.ObjectMapper
import com.typesafe.scalalogging.Logger
import org.apache.spark.{SparkContext,SparkConf}
import org.slf4j.LoggerFactory
import scala.collection.mutable.ArrayBuffer
class Person {
    @JsonProperty var first:String=null
    @JsonProperty var last:String=null
    @JsonProperty var address:Address=null
    override def toString=s"Person(first=$first, last=$last, address=$address)"
}

class Address {
    @JsonProperty var line1:String=null
    @JsonProperty var line2:String=null
    @JsonProperty var city:String=null
    @JsonProperty var state:String=null
    @JsonProperty var zip:String=null
    override def toString=s"Address(line1=$line1, line2=$line2, city=$city, state=$state, zip=$zip)"
}

object JSONFileReaderWriter{
    // val logger =     Logger(LoggerFactory.getLogger("JSONFileReaderWriter"))
    val mapper = new ObjectMapper()

    def main(argv: Array[String]): Unit = {

    if (argv.length != 2) {
        println("Please provide 2 parameters &lt;inputfile&gt; &lt;outputfile&gt;")
        System.exit(1)
    }
    val inputFile = argv(0)
    val outputFile = argv(1)

    println(inputFile)
    println(outputFile)

    //logger.debug(s"Read json from $inputFile and write to $outputFile")

    val sparkConf = new   SparkConf().setMaster("local[1]").setAppName("JSONFileReaderWriter")
    val sparkContext = new SparkContext(sparkConf)
    val errorRecords = sparkContext.accumulator(0)
    val records = sparkContext.textFile(inputFile)

    var results = records.flatMap { record =&gt;
        try {
            Some(mapper.readValue(record, classOf[Person]))
        } catch {
            case e: Exception =&gt; {
            errorRecords += 1
            None
        }
     }
   }//.filter(person =&gt; person.address.city.equals("mumbai"))

   results.saveAsTextFile(outputFile)

println("Number of bad records " + errorRecords)
   }
}
</code></pre>

<p>But when there is a JSONArray inside the JSONObject, I could not figure it out how to extend the code. Any help is really appreciated.</p>

<p>Here is the JSONObject that I want to read to RDD without using dataframe:</p>

<pre><code>{"first":"John","last":"Smith","address":[{"line1":"1 main street","city":"San Francisco","state":"CA","zip":"94101"},{"line1":"2 main street","city":"Palo Alto","state":"CA","zip":"94305"}]}
</code></pre>

<p>I DO NOT want to use Spark SQL.</p>