{"action":"created","issue":{"url":"https://api.github.com/repos/elastic/elasticsearch/issues/14442","labels_url":"https://api.github.com/repos/elastic/elasticsearch/issues/14442/labels{/name}","comments_url":"https://api.github.com/repos/elastic/elasticsearch/issues/14442/comments","events_url":"https://api.github.com/repos/elastic/elasticsearch/issues/14442/events","html_url":"https://github.com/elastic/elasticsearch/issues/14442","id":114631760,"number":14442,"title":"Elasticsearch _bulk endpoint fails if data contains UTF-8 Byte Order Mark","user":{"login":"jprante","id":635745,"avatar_url":"https://avatars.githubusercontent.com/u/635745?v=3","gravatar_id":"","url":"https://api.github.com/users/jprante","html_url":"https://github.com/jprante","followers_url":"https://api.github.com/users/jprante/followers","following_url":"https://api.github.com/users/jprante/following{/other_user}","gists_url":"https://api.github.com/users/jprante/gists{/gist_id}","starred_url":"https://api.github.com/users/jprante/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jprante/subscriptions","organizations_url":"https://api.github.com/users/jprante/orgs","repos_url":"https://api.github.com/users/jprante/repos","events_url":"https://api.github.com/users/jprante/events{/privacy}","received_events_url":"https://api.github.com/users/jprante/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/elastic/elasticsearch/labels/:REST","name":":REST","color":"009800"},{"url":"https://api.github.com/repos/elastic/elasticsearch/labels/adoptme","name":"adoptme","color":"207de5"},{"url":"https://api.github.com/repos/elastic/elasticsearch/labels/low%20hanging%20fruit","name":"low hanging fruit","color":"009800"},{"url":"https://api.github.com/repos/elastic/elasticsearch/labels/v2.0.1","name":"v2.0.1","color":"dddddd"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2015-11-02T16:32:28Z","updated_at":"2015-11-09T10:42:20Z","closed_at":null,"body":"This looks like a regression because it worked on 1.x\r\n\r\nperl -e 'print \"\\xEF\\xBB\\xBF\"' > test.dat\r\necho '{ \"index\" : { \"_index\" : \"test\", \"_type\" : \"type1\", \"_id\" : \"1\" } }' >> test.dat\r\necho '{ \"field1\" : \"value1\" }' >> test.dat\r\n\r\nhexdump -C test.dat \r\n\r\n```\r\n00000000  ef bb bf 7b 20 22 69 6e  64 65 78 22 20 3a 20 7b  |...{ \"index\" : {|\r\n00000010  20 22 5f 69 6e 64 65 78  22 20 3a 20 22 74 65 73  | \"_index\" : \"tes|\r\n00000020  74 22 2c 20 22 5f 74 79  70 65 22 20 3a 20 22 74  |t\", \"_type\" : \"t|\r\n00000030  79 70 65 31 22 2c 20 22  5f 69 64 22 20 3a 20 22  |ype1\", \"_id\" : \"|\r\n00000040  31 22 20 7d 20 7d 0a 7b  20 22 66 69 65 6c 64 31  |1\" } }.{ \"field1|\r\n00000050  22 20 3a 20 22 76 61 6c  75 65 31 22 20 7d 0a     |\" : \"value1\" }.|\r\n0000005f\r\n```\r\n\r\nElasticsearch 2.0.0\r\n\r\n```\r\n[2015-11-02 17:20:49,328][INFO ][node                     ] [Earthquake] version[2.0.0], pid[77392], build[de54438/2015-10-22T08:09:48Z]\r\n```\r\ncurl -XPOST '0:9200/_bulk' --data-binary @test.dat\r\n\r\nfails with \r\n\r\n```\r\n{\"error\":{\"root_cause\":[{\"type\":\"parse_exception\",\"reason\":\"Failed to derive xcontent\"}],\"type\":\"parse_exception\",\"reason\":\"Failed to derive xcontent\"},\"status\":400}\r\n```\r\n\r\nServer log\r\n\r\n```\r\n[2015-11-02 17:21:58,254][INFO ][rest.suppressed          ] /_bulk Params: {}\r\nElasticsearchParseException[Failed to derive xcontent]\r\n\tat org.elasticsearch.common.xcontent.XContentFactory.xContent(XContentFactory.java:293)\r\n\tat org.elasticsearch.action.bulk.BulkRequest.add(BulkRequest.java:251)\r\n\tat org.elasticsearch.rest.action.bulk.RestBulkAction.handleRequest(RestBulkAction.java:88)\r\n\tat org.elasticsearch.rest.BaseRestHandler.handleRequest(BaseRestHandler.java:54)\r\n\tat org.elasticsearch.rest.RestController.executeHandler(RestController.java:207)\r\n\tat org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:166)\r\n\tat org.elasticsearch.http.HttpServer.internalDispatchRequest(HttpServer.java:128)\r\n\tat org.elasticsearch.http.HttpServer$Dispatcher.dispatchRequest(HttpServer.java:86)\r\n\tat org.elasticsearch.http.netty.NettyHttpServerTransport.dispatchRequest(NettyHttpServerTransport.java:348)\r\n\tat org.elasticsearch.http.netty.HttpRequestHandler.messageReceived(HttpRequestHandler.java:63)\r\n\tat org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)\r\n\tat org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)\r\n\tat org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)\r\n\tat org.elasticsearch.http.netty.pipelining.HttpPipeliningHandler.messageReceived(HttpPipeliningHandler.java:60)\r\n\tat org.jboss.netty.channel.SimpleChannelHandler.handleUpstream(SimpleChannelHandler.java:88)\r\n\tat org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)\r\n\tat org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)\r\n\tat org.jboss.netty.handler.codec.http.HttpChunkAggregator.messageReceived(HttpChunkAggregator.java:145)\r\n\tat org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)\r\n\tat org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)\r\n\tat org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)\r\n\tat org.jboss.netty.handler.codec.http.HttpContentDecoder.messageReceived(HttpContentDecoder.java:108)\r\n\tat org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)\r\n\tat org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)\r\n\tat org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)\r\n\tat org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)\r\n\tat org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:459)\r\n\tat org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:536)\r\n\tat org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:435)\r\n\tat org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)\r\n\tat org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)\r\n\tat org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)\r\n\tat org.elasticsearch.common.netty.OpenChannelsHandler.handleUpstream(OpenChannelsHandler.java:75)\r\n\tat org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)\r\n\tat org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)\r\n\tat org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)\r\n\tat org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)\r\n\tat org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)\r\n\tat org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)\r\n\tat org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)\r\n\tat org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)\r\n\tat org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)\r\n\tat org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)\r\n\tat org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)\r\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)\r\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)\r\n\tat java.lang.Thread.run(Thread.java:745)\r\n```"},"comment":{"url":"https://api.github.com/repos/elastic/elasticsearch/issues/comments/155025529","html_url":"https://github.com/elastic/elasticsearch/issues/14442#issuecomment-155025529","issue_url":"https://api.github.com/repos/elastic/elasticsearch/issues/14442","id":155025529,"user":{"login":"jpountz","id":299848,"avatar_url":"https://avatars.githubusercontent.com/u/299848?v=3","gravatar_id":"","url":"https://api.github.com/users/jpountz","html_url":"https://github.com/jpountz","followers_url":"https://api.github.com/users/jpountz/followers","following_url":"https://api.github.com/users/jpountz/following{/other_user}","gists_url":"https://api.github.com/users/jpountz/gists{/gist_id}","starred_url":"https://api.github.com/users/jpountz/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jpountz/subscriptions","organizations_url":"https://api.github.com/users/jpountz/orgs","repos_url":"https://api.github.com/users/jpountz/repos","events_url":"https://api.github.com/users/jpountz/events{/privacy}","received_events_url":"https://api.github.com/users/jpountz/received_events","type":"User","site_admin":false},"created_at":"2015-11-09T10:42:20Z","updated_at":"2015-11-09T10:42:20Z","body":"@camilojd No need to send multiple PRs, we will take care of backporting."}}