Read multiline JSON in Apache Spark
<p>I was trying to use a JSON file as a small DB. After creating a template table on DataFrame I queried it with SQL and got an exception. Here is my code:</p>

<pre><code>val df = sqlCtx.read.json("/path/to/user.json")
df.registerTempTable("user_tt")

val info = sqlCtx.sql("SELECT name FROM user_tt")
info.show()
</code></pre>

<p><code>df.printSchema()</code> result:</p>

<pre><code>root
 |-- _corrupt_record: string (nullable = true)
</code></pre>

<p>My JSON file:</p>

<pre><code>{
  "id": 1,
  "name": "Morty",
  "age": 21
}
</code></pre>

<p>Exeption:</p>

<pre><code>Exception in thread "main" org.apache.spark.sql.AnalysisException: cannot resolve 'name' given input columns: [_corrupt_record];
</code></pre>

<p>How can I fix it?</p>

<p><em>UPD</em></p>

<p><code>_corrupt_record</code> is</p>

<pre><code>+--------------------+
|     _corrupt_record|
+--------------------+
|                   {|
|            "id": 1,|
|    "name": "Morty",|
|           "age": 21|
|                   }|
+--------------------+
</code></pre>

<p><em>UPD2</em></p>

<p>It's weird, but when I rewrite my JSON to make it oneliner, everything works fine.</p>

<pre><code>{"id": 1, "name": "Morty", "age": 21}
</code></pre>

<p>So the problem is in a <code>newline</code>.</p>

<p><em>UPD3</em></p>

<p>I found in docs the next sentence:</p>

<blockquote>
  <p>Note that the file that is offered as a json file is not a typical JSON file. Each line must contain a separate, self-contained valid JSON object. As a consequence, a regular multi-line JSON file will most often fail.</p>
</blockquote>

<p>It isn't convenient to keep JSON in such format. Is there any workaround to get rid of multi-lined structure of JSON or to convert it in oneliner?</p>