Defining DateType conversion for DataFrame schema in Spark
<p>I am reading DataFrame from CSV file, where first column is an event date and time e.g.</p>

<blockquote>
  <p>2016-08-08 07:45:28+03</p>
</blockquote>

<p>In code below, is it possible to specify within schema definition how to convert such strings into date?</p>

<pre><code>val df:DataFrame = spark.read.options(Map(
  "header" -&gt; "true"
)).schema(StructType(
    StructField("EventTime", DataTypes.DateType, false) ::
    Nil
)).csv("C:/qos1h.csv")
</code></pre>

<p>This code fails with</p>

<pre><code>java.lang.NumberFormatException: For input string: "28+03"
    at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
    at java.lang.Integer.parseInt(Integer.java:580)
    at java.lang.Integer.parseInt(Integer.java:615)
    at java.sql.Timestamp.valueOf(Timestamp.java:259)
    at org.apache.spark.sql.catalyst.util.DateTimeUtils$.stringToTime(DateTimeUtils.scala:135)
    at org.apache.spark.sql.execution.datasources.csv.CSVTypeCast$.castTo(CSVInferSchema.scala:291)
    at org.apache.spark.sql.execution.datasources.csv.CSVRelation$$anonfun$csvParser$3.apply(CSVRelation.scala:115)
    at org.apache.spark.sql.execution.datasources.csv.CSVRelation$$anonfun$csvParser$3.apply(CSVRelation.scala:84)
    at org.apache.spark.sql.execution.datasources.csv.CSVFileFormat$$anonfun$buildReader$1$$anonfun$apply$1.apply(CSVFileFormat.scala:125)
    at org.apache.spark.sql.execution.datasources.csv.CSVFileFormat$$anonfun$buildReader$1$$anonfun$apply$1.apply(CSVFileFormat.scala:124)
</code></pre>