{"action":"created","issue":{"url":"https://api.github.com/repos/elastic/elasticsearch-hadoop/issues/565","labels_url":"https://api.github.com/repos/elastic/elasticsearch-hadoop/issues/565/labels{/name}","comments_url":"https://api.github.com/repos/elastic/elasticsearch-hadoop/issues/565/comments","events_url":"https://api.github.com/repos/elastic/elasticsearch-hadoop/issues/565/events","html_url":"https://github.com/elastic/elasticsearch-hadoop/issues/565","id":110258255,"number":565,"title":"Is it possible to write to Amazon Elasticsearch Service using elasticsearch-hadoop?","user":{"login":"fblundun","id":5892914,"avatar_url":"https://avatars.githubusercontent.com/u/5892914?v=3","gravatar_id":"","url":"https://api.github.com/users/fblundun","html_url":"https://github.com/fblundun","followers_url":"https://api.github.com/users/fblundun/followers","following_url":"https://api.github.com/users/fblundun/following{/other_user}","gists_url":"https://api.github.com/users/fblundun/gists{/gist_id}","starred_url":"https://api.github.com/users/fblundun/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fblundun/subscriptions","organizations_url":"https://api.github.com/users/fblundun/orgs","repos_url":"https://api.github.com/users/fblundun/repos","events_url":"https://api.github.com/users/fblundun/events{/privacy}","received_events_url":"https://api.github.com/users/fblundun/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/elastic/elasticsearch-hadoop/labels/cascading","name":"cascading","color":"e102d8"},{"url":"https://api.github.com/repos/elastic/elasticsearch-hadoop/labels/question","name":"question","color":"cc317c"},{"url":"https://api.github.com/repos/elastic/elasticsearch-hadoop/labels/rest","name":"rest","color":"DDDDDD"},{"url":"https://api.github.com/repos/elastic/elasticsearch-hadoop/labels/v2.1.2","name":"v2.1.2","color":"bfdadc"},{"url":"https://api.github.com/repos/elastic/elasticsearch-hadoop/labels/v2.2.0-m2","name":"v2.2.0-m2","color":"bfdadc"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":10,"created_at":"2015-10-07T15:56:29Z","updated_at":"2015-10-22T18:07:07Z","closed_at":null,"body":"I have been trying to write to AWS's new Amazon Elasticsearch Service from a Scalding job using elasticsearch-hadoop (via [scalding-taps](https://github.com/scalding-io/scalding-taps)).\r\n\r\nThis job has previously worked using Elasticsearch manually installed on an EC2 instance, implying that the problem is specific to Amazon Elasticsearch Service.\r\n\r\nThe first time I tried I got this error:\r\n\r\n```\r\nCaused by: org.elasticsearch.hadoop.EsHadoopIllegalStateException: Cluster state volatile; cannot find node backing shards - please check whether your cluster is stable\r\n        at org.elasticsearch.hadoop.rest.RestRepository.getWriteTargetPrimaryShards(RestRepository.java:370)\r\n        at org.elasticsearch.hadoop.rest.RestService.initSingleIndex(RestService.java:425)\r\n        at org.elasticsearch.hadoop.rest.RestService.createWriter(RestService.java:393)\r\n        at org.elasticsearch.hadoop.mr.EsOutputFormat$EsRecordWriter.init(EsOutputFormat.java:173)\r\n        at org.elasticsearch.hadoop.mr.EsOutputFormat$EsRecordWriter.write(EsOutputFormat.java:149)\r\n        at org.apache.hadoop.mapred.MapTask$DirectMapOutputCollector.collect(MapTask.java:844)\r\n        at org.apache.hadoop.mapred.MapTask$OldOutputCollector.collect(MapTask.java:596)\r\n        at cascading.tap.hadoop.util.MeasuredOutputCollector.collect(MeasuredOutputCollector.java:69)\r\n        at org.elasticsearch.hadoop.cascading.EsHadoopScheme.sink(EsHadoopScheme.java:212)\r\n        at cascading.tuple.TupleEntrySchemeCollector.collect(TupleEntrySchemeCollector.java:153)\r\n        ... 21 more\r\n```\r\n\r\nI then tried setting `es.nodes.client.only` to `true` and instead got this error:\r\n\r\n```\r\nCaused by: org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: Client-only routing specified but no client nodes with HTTP-enabled were found in the cluster...\r\n        at org.elasticsearch.hadoop.rest.InitializationUtils.filterNonClientNodesIfNeeded(InitializationUtils.java:82)\r\n        at org.elasticsearch.hadoop.rest.RestService.createWriter(RestService.java:373)\r\n        at org.elasticsearch.hadoop.mr.EsOutputFormat$EsRecordWriter.init(EsOutputFormat.java:173)\r\n        at org.elasticsearch.hadoop.mr.EsOutputFormat$EsRecordWriter.write(EsOutputFormat.java:149)\r\n        at org.apache.hadoop.mapred.MapTask$DirectMapOutputCollector.collect(MapTask.java:844)\r\n        at org.apache.hadoop.mapred.MapTask$OldOutputCollector.collect(MapTask.java:596)\r\n        at cascading.tap.hadoop.util.MeasuredOutputCollector.collect(MeasuredOutputCollector.java:69)\r\n        at org.elasticsearch.hadoop.cascading.EsHadoopScheme.sink(EsHadoopScheme.java:212)\r\n        at cascading.tuple.TupleEntrySchemeCollector.collect(TupleEntrySchemeCollector.java:153)\r\n        ... 21 more\r\n```\r\n\r\nI haven't made any changes to Amazon's default Elasticsearch configuration. Does anybody have any idea how to make this work?"},"comment":{"url":"https://api.github.com/repos/elastic/elasticsearch-hadoop/issues/comments/150308511","html_url":"https://github.com/elastic/elasticsearch-hadoop/issues/565#issuecomment-150308511","issue_url":"https://api.github.com/repos/elastic/elasticsearch-hadoop/issues/565","id":150308511,"user":{"login":"fblundun","id":5892914,"avatar_url":"https://avatars.githubusercontent.com/u/5892914?v=3","gravatar_id":"","url":"https://api.github.com/users/fblundun","html_url":"https://github.com/fblundun","followers_url":"https://api.github.com/users/fblundun/followers","following_url":"https://api.github.com/users/fblundun/following{/other_user}","gists_url":"https://api.github.com/users/fblundun/gists{/gist_id}","starred_url":"https://api.github.com/users/fblundun/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fblundun/subscriptions","organizations_url":"https://api.github.com/users/fblundun/orgs","repos_url":"https://api.github.com/users/fblundun/repos","events_url":"https://api.github.com/users/fblundun/events{/privacy}","received_events_url":"https://api.github.com/users/fblundun/received_events","type":"User","site_admin":false},"created_at":"2015-10-22T18:07:07Z","updated_at":"2015-10-22T18:07:07Z","body":"Thanks very much @costin! I will try it out tomorrow and report the results."}}