Spark SQL: Sqlite - Unparseable date
<p>I try to process a database with Apache Spark. But I get this <em>unparseable date exception</em>. I read all the other questions about this exception here on SO, but they don't help me. The interesing part of my code is this:</p>

<pre><code>try {
  val tableData = sqlContext.read.format("jdbc")
                            .options(Map("url" -&gt; databasePath,
                                         "dbtable" -&gt; t))
                            .load()
  if (tableData.count &gt; 0) {
    val df = tableData.show                    
  }

} catch {
  case s: SQLException =&gt; println("DEBUG, SKIPPING TABLE " + t)
  case e: Exception =&gt; println("EXCEPTION CAUGHT: " + t); System.exit(1)
}
</code></pre>

<p>So although I get <code>java.sql.SQLException: Error parsing date</code> I cannot catch this exception, it always takes the second case statement.</p>

<p>While catching the exception and just skipping the table would be a good start I am more interested in making it actually work. But I never call <code>Date.parse</code> manually so I don't know how to apply the answers from the other questions.</p>

<p>More output:</p>

<pre><code>Caused by: java.text.ParseException: Unparseable date: "2009-01-01 00:00:00" does not match (\p{Nd}++)\Q-\E(\p{Nd}++)\Q-\E(\p{Nd}++)\Q \E(\p{Nd}++)\Q:\E(\p{Nd}++)\Q:\E(\p{Nd}++)\Q.\E(\p{Nd}++)
</code></pre>

<p>Edit1: As requested, schema and example data:</p>

<pre><code>CREATE TABLE [Invoice]
(
  [InvoiceId] INTEGER  NOT NULL,
  [CustomerId] INTEGER  NOT NULL,
  [InvoiceDate] DATETIME  NOT NULL,
  [BillingAddress] NVARCHAR(70),
  [BillingCity] NVARCHAR(40),
  [BillingState] NVARCHAR(40),
  [BillingCountry] NVARCHAR(40),
  [BillingPostalCode] NVARCHAR(10),
  [Total] NUMERIC(10,2)  NOT NULL,
  CONSTRAINT [PK_Invoice] PRIMARY KEY  ([InvoiceId]),
  FOREIGN KEY ([CustomerId]) REFERENCES [Customer] ([CustomerId]) 
    ON DELETE NO ACTION ON UPDATE NO ACTION
);

SELECT * FROM Invoice LIMIT 5;
1|2|2009-01-01 00:00:00|Theodor-Heuss-Straße 34|Stuttgart||Germany|70174|1.98
2|4|2009-01-02 00:00:00|Ullevålsveien 14|Oslo||Norway|0171|3.96
3|8|2009-01-03 00:00:00|Grétrystraat 63|Brussels||Belgium|1000|5.94
4|14|2009-01-06 00:00:00|8210 111 ST NW|Edmonton|AB|Canada|T6G 2C7|8.91
5|23|2009-01-11 00:00:00|69 Salem Street|Boston|MA|USA|2113|13.86
</code></pre>

<p>Edit2: I am running Spark 1.5.1. Further investigation lead me to this commit: <a href="https://github.com/apache/spark/commit/d39f15ea2b8bed5342d2f8e3c1936f915c470783" rel="nofollow">https://github.com/apache/spark/commit/d39f15ea2b8bed5342d2f8e3c1936f915c470783</a>. The test cases clearly include the format which is used in my table. But this is a commit for Spark 1.6.0 and I first have to check all my dependecies if it is safe to update.</p>