{"action":"closed","issue":{"url":"https://api.github.com/repos/logstash-plugins/logstash-filter-csv/issues/16","labels_url":"https://api.github.com/repos/logstash-plugins/logstash-filter-csv/issues/16/labels{/name}","comments_url":"https://api.github.com/repos/logstash-plugins/logstash-filter-csv/issues/16/comments","events_url":"https://api.github.com/repos/logstash-plugins/logstash-filter-csv/issues/16/events","html_url":"https://github.com/logstash-plugins/logstash-filter-csv/issues/16","id":113698473,"number":16,"title":"Issues with numerical values that contain commas","user":{"login":"sirmodok","id":6287669,"avatar_url":"https://avatars.githubusercontent.com/u/6287669?v=3","gravatar_id":"","url":"https://api.github.com/users/sirmodok","html_url":"https://github.com/sirmodok","followers_url":"https://api.github.com/users/sirmodok/followers","following_url":"https://api.github.com/users/sirmodok/following{/other_user}","gists_url":"https://api.github.com/users/sirmodok/gists{/gist_id}","starred_url":"https://api.github.com/users/sirmodok/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sirmodok/subscriptions","organizations_url":"https://api.github.com/users/sirmodok/orgs","repos_url":"https://api.github.com/users/sirmodok/repos","events_url":"https://api.github.com/users/sirmodok/events{/privacy}","received_events_url":"https://api.github.com/users/sirmodok/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/logstash-plugins/logstash-filter-csv/labels/needs%20details","name":"needs details","color":"cc317c"}],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2015-10-27T21:55:27Z","updated_at":"2015-11-30T19:03:40Z","closed_at":"2015-11-30T19:03:40Z","body":"When I use the csv filter to parse a large CSV file, I see some issues where numerical values that contain a comma break the parsing despite being in double quotes. \r\nMy filter looks something like this\r\n\r\n```\r\nfilter {\r\n  if [type] == \"nexpose\" {\r\n    # Uses the csv plugin to parse information from a CSV file into elasticsearch\r\n    csv {\r\n      columns => [\"color\", \"shoe\", \"model\", \"Asset Risk Score\",\"horses name\",\"Asset OS Name\", \"small number\", \"fav ghostbuster\" [...]\r\n      separator => \",\"\r\n    }\r\n    grok {\r\n      match => {\"color\" => \"%{TIME:timestamp2}Z %{HOSTNAME:host} %{GREEDYDATA:colour}\"}\r\n    }\r\n   # remove some useless fields\r\n    mutate {\r\n      remove_field => [ \"timestamp2\", \"color\" ]\r\n    }\r\n    # Drop the top row of the CSV file\r\n    if [shoe] == \"shoe\" { drop{ } }\r\n  }\r\n}\r\n```\r\n\r\nIf I feed it a CSV that contains a line that looks like this\r\n\r\n```\r\npurple,nike,Heidi Klum,\"133,327\",\"Microsoft Windows Server 2008 R2, Standard Edition\",America,23,Venkman\r\n```\r\n\r\nThen in the results for the field \"Asset Risk Score\" will be \"133\" and not \"133,327\" even if the result for the field \"Asset OS Name\" is correctly \"Microsoft Windows Server 2008 R2, Standard Edition\"\r\n\r\n![image](https://cloud.githubusercontent.com/assets/6287669/10773170/e4dcc57a-7cc7-11e5-80d7-79ad598a3cf6.png)\r\n\r\nBoth values are in double quotes, the only difference is that one is numerical and the other is a string. \r\n\r\nHere is an example from the logstash logs viewpoint\r\n![image](https://cloud.githubusercontent.com/assets/6287669/10773763/487dca5e-7ccb-11e5-815c-3dd63a7757f2.png)\r\n\r\nHas anyone else seen anything like this?\r\n\r\n"}}