{"action":"created","issue":{"url":"https://api.github.com/repos/databricks/spark-csv/issues/105","labels_url":"https://api.github.com/repos/databricks/spark-csv/issues/105/labels{/name}","comments_url":"https://api.github.com/repos/databricks/spark-csv/issues/105/comments","events_url":"https://api.github.com/repos/databricks/spark-csv/issues/105/events","html_url":"https://github.com/databricks/spark-csv/pull/105","id":96620312,"number":105,"title":"Support comment characters","user":{"login":"jaley","id":343449,"avatar_url":"https://avatars.githubusercontent.com/u/343449?v=3","gravatar_id":"","url":"https://api.github.com/users/jaley","html_url":"https://github.com/jaley","followers_url":"https://api.github.com/users/jaley/followers","following_url":"https://api.github.com/users/jaley/following{/other_user}","gists_url":"https://api.github.com/users/jaley/gists{/gist_id}","starred_url":"https://api.github.com/users/jaley/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jaley/subscriptions","organizations_url":"https://api.github.com/users/jaley/orgs","repos_url":"https://api.github.com/users/jaley/repos","events_url":"https://api.github.com/users/jaley/events{/privacy}","received_events_url":"https://api.github.com/users/jaley/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":6,"created_at":"2015-07-22T17:19:43Z","updated_at":"2015-07-30T13:17:48Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/databricks/spark-csv/pulls/105","html_url":"https://github.com/databricks/spark-csv/pull/105","diff_url":"https://github.com/databricks/spark-csv/pull/105.diff","patch_url":"https://github.com/databricks/spark-csv/pull/105.patch"},"body":"Many automatically generated TSV files, such as logs from [Amazon CloudFront](http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/AccessLogs.html) use a `#` character for comments (used to put version information or sometimes even headers) at the top of the file. The CSV parser actually supports this already, so this change just connects it up to the parameters.\r\n\r\nThe requirement to handle comments at the start of the file is actually a bit problematic, because CsvReation actually peeks at the first line to infer the schema if it's not explicitly provided. The implementation I went with was to pick some fixed maximum number of lines we will peek ahead searching for a line that isn't a comment before failing, if commentMarker is provided. Not entirely happy with that as it feels a bit nasty, but not sure what else could be done - suggestions welcome!\r\n"},"comment":{"url":"https://api.github.com/repos/databricks/spark-csv/issues/comments/126320154","html_url":"https://github.com/databricks/spark-csv/pull/105#issuecomment-126320154","issue_url":"https://api.github.com/repos/databricks/spark-csv/issues/105","id":126320154,"user":{"login":"jaley","id":343449,"avatar_url":"https://avatars.githubusercontent.com/u/343449?v=3","gravatar_id":"","url":"https://api.github.com/users/jaley","html_url":"https://github.com/jaley","followers_url":"https://api.github.com/users/jaley/followers","following_url":"https://api.github.com/users/jaley/following{/other_user}","gists_url":"https://api.github.com/users/jaley/gists{/gist_id}","starred_url":"https://api.github.com/users/jaley/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jaley/subscriptions","organizations_url":"https://api.github.com/users/jaley/orgs","repos_url":"https://api.github.com/users/jaley/repos","events_url":"https://api.github.com/users/jaley/events{/privacy}","received_events_url":"https://api.github.com/users/jaley/received_events","type":"User","site_admin":false},"created_at":"2015-07-30T13:17:48Z","updated_at":"2015-07-30T13:17:48Z","body":"@falaki OK, I fixed that problem now. It's now possible to disable comments by setting the parameter to null. The default is `#` for both parsers."}}