{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/6057","labels_url":"https://api.github.com/repos/apache/spark/issues/6057/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/6057/comments","events_url":"https://api.github.com/repos/apache/spark/issues/6057/events","html_url":"https://github.com/apache/spark/pull/6057","id":75262287,"number":6057,"title":"[SPARK-7278] [PySpark] DateType should find datetime.datetime acceptable","user":{"login":"ksonj","id":7082519,"avatar_url":"https://avatars.githubusercontent.com/u/7082519?v=3","gravatar_id":"","url":"https://api.github.com/users/ksonj","html_url":"https://github.com/ksonj","followers_url":"https://api.github.com/users/ksonj/followers","following_url":"https://api.github.com/users/ksonj/following{/other_user}","gists_url":"https://api.github.com/users/ksonj/gists{/gist_id}","starred_url":"https://api.github.com/users/ksonj/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ksonj/subscriptions","organizations_url":"https://api.github.com/users/ksonj/orgs","repos_url":"https://api.github.com/users/ksonj/repos","events_url":"https://api.github.com/users/ksonj/events{/privacy}","received_events_url":"https://api.github.com/users/ksonj/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2015-05-11T15:34:52Z","updated_at":"2015-05-11T19:14:45Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/6057","html_url":"https://github.com/apache/spark/pull/6057","diff_url":"https://github.com/apache/spark/pull/6057.diff","patch_url":"https://github.com/apache/spark/pull/6057.patch"},"body":"DateType should not be restricted to `datetime.date` but accept `datetime.datetime` objects as well. Could someone with a little more insight verify this?"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/101020312","html_url":"https://github.com/apache/spark/pull/6057#issuecomment-101020312","issue_url":"https://api.github.com/repos/apache/spark/issues/6057","id":101020312,"user":{"login":"ksonj","id":7082519,"avatar_url":"https://avatars.githubusercontent.com/u/7082519?v=3","gravatar_id":"","url":"https://api.github.com/users/ksonj","html_url":"https://github.com/ksonj","followers_url":"https://api.github.com/users/ksonj/followers","following_url":"https://api.github.com/users/ksonj/following{/other_user}","gists_url":"https://api.github.com/users/ksonj/gists{/gist_id}","starred_url":"https://api.github.com/users/ksonj/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ksonj/subscriptions","organizations_url":"https://api.github.com/users/ksonj/orgs","repos_url":"https://api.github.com/users/ksonj/repos","events_url":"https://api.github.com/users/ksonj/events{/privacy}","received_events_url":"https://api.github.com/users/ksonj/received_events","type":"User","site_admin":false},"created_at":"2015-05-11T19:14:45Z","updated_at":"2015-05-11T19:14:45Z","body":"There's an inconsistency in how date/datetimes are handled in pyspark. Dates will always be deserialized as `datetime` but will not always be converted back. See also the referenced JIRA. This makes the following code complain about `datetime.datetime` not being an acceptable `DateType`\r\n\r\n    sq = pyspark.sql.SQLContext(sc)\r\n    df = sq.parquetFile(src)\r\n    schema = df.schema\r\n    sq.createDataFrame(df.rdd, schema=schema)"}}