How to troubleshoot java.lang.NumberFormatException: null
<p>I am loading a file that has ~500,000 records such as this</p>

<pre><code>ROW_ID, COLOR_CODE, SHADE_ID
21, 22, 321
23, 31, 321
</code></pre>

<p>I load it like this:</p>

<pre><code> val colorSchema = StructType(Array(
         StructField("ROW_ID", IntegerType, true),
         StructField("COLOR_CODE", IntegerType, true),
         StructField("SHADE_ID", IntegerType, true)

     def makeSchema(filename:String, tableName:String,
         tableSchema:StructType,uri:String){

         val table = spark.read.
           format("com.databricks.spark.csv").
           option("header", "true").
           schema(tableSchema).load(uri+filename).cache()
         table.registerTempTable(tableName.toUpperCase)
       }

makeSchema("colors.csv","colors",colorSchema,"s3://bucket/")
</code></pre>

<p>The above code runs fine. However, when I run the following query I get an error <code>java.lang.NumberFormatException: null</code></p>

<pre><code>val r = spark.sql("select * from colors where COLOR_CODE = 22").take(1)
</code></pre>

<p>What am I doing wrong? And how can I spot this issue in an effective way? I have visually scanned the file to see if <code>COLOR_CODE</code> has missing values but I can't see any visually...</p>

<p><strong>Update</strong></p>

<p>I've asked a separate question that narrows down the problem further. The CSV now only has 1 row and I still get the same error. <a href="https://stackoverflow.com/questions/40564201/how-to-resolve-java-lang-numberformatexception-null-in-spark-sql">https://stackoverflow.com/questions/40564201/how-to-resolve-java-lang-numberformatexception-null-in-spark-sql</a></p>