Parsing datetime from ISO 8601 using Spark SQL
<p>Want to do <a href="https://stackoverflow.com/questions/9321809/format-date-in-mysql-select-as-iso-8601">this</a> but the other way around. </p>

<p>My <code>date</code>s are in this format <code>YYYY-MM-DDThh:mm:ss</code>, I want two columns <code>YYYY-MM-DD</code> and <code>hh:mm</code> that I can concat, if I want to, for certain queries. </p>

<p>I get an error when using <code>convert()</code>; I assume this is not supported currently with Spark SQL. </p>

<p>When I use <code>date(datetime)</code> or <code>timestamp(datetime)</code>, I get all null values returned. However, <code>minute(datetime)</code> and <code>hour(datetime)</code> work.</p>

<p>Currently, using this</p>

<pre><code>concat(date,' ', hour,':', (case when minute &lt; 10 then concat('0',minute) else minute end)) as DateTime
from (select OtherDateOnlyColumn as date, minute(datetime) as minute, hour(datetime) as hour from ...)
</code></pre>

<p>which is obviously not efficient. </p>