{"action":"created","issue":{"url":"https://api.github.com/repos/elastic/elasticsearch/issues/10839","labels_url":"https://api.github.com/repos/elastic/elasticsearch/issues/10839/labels{/name}","comments_url":"https://api.github.com/repos/elastic/elasticsearch/issues/10839/comments","events_url":"https://api.github.com/repos/elastic/elasticsearch/issues/10839/events","html_url":"https://github.com/elastic/elasticsearch/issues/10839","id":71397496,"number":10839,"title":"bug in bulk java api?","user":{"login":"rrphotosoft","id":4465221,"avatar_url":"https://avatars.githubusercontent.com/u/4465221?v=3","gravatar_id":"","url":"https://api.github.com/users/rrphotosoft","html_url":"https://github.com/rrphotosoft","followers_url":"https://api.github.com/users/rrphotosoft/followers","following_url":"https://api.github.com/users/rrphotosoft/following{/other_user}","gists_url":"https://api.github.com/users/rrphotosoft/gists{/gist_id}","starred_url":"https://api.github.com/users/rrphotosoft/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rrphotosoft/subscriptions","organizations_url":"https://api.github.com/users/rrphotosoft/orgs","repos_url":"https://api.github.com/users/rrphotosoft/repos","events_url":"https://api.github.com/users/rrphotosoft/events{/privacy}","received_events_url":"https://api.github.com/users/rrphotosoft/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":6,"created_at":"2015-04-27T20:46:44Z","updated_at":"2015-04-28T01:19:20Z","closed_at":null,"body":"Hi, \r\n\r\nI have noticed what looks like a small bug in the java bulk api for ES.\r\nI'm not sure if this is the right place to post it but here goes anyways:\r\n\r\nAssume that we set the bulk size to execute every 10 requests.\r\nThe expected response is that the afterBulk Callback should be called, once the response is executed.\r\nUnfortunately if I add just 10 items to the bulk queue, there is no response, i.e the afterBulk Callback is not triggered.\r\nI can see that the items are added through sense, or even in the ES console.\r\n\r\nIn case I add more than 10 items, I get responses for everything after the first 10.\r\n\r\nCode to reproduce the issue:\r\nN.B:\r\n*ElasticClientHolder in the code below, is a singleton holding a connection to ES\r\n\r\nElasticsearch v - 1.4.4\r\nUbuntu 12.04 LTS\r\n\r\n      public class BulkProcessorTestClasses {\r\n\t private BulkProcessor bulkprocessor;\r\n\t private BulkRequestBuilder builder;\r\n\t\r\n\tpublic BulkProcessor getBulkprocessor() {\r\n\t\tif (bulkprocessor == null) {\r\n\t\t\tthis.bulkprocessor = BulkProcessor\r\n\t\t\t\t\t.builder(ElasticClientHolder.getInstance(),\r\n\t\t\t\t\t\t\tnew BulkProcessor.Listener() {\r\n\r\n\t\t\t\t\t\t\t\t@Override\r\n\t\t\t\t\t\t\t\tpublic void afterBulk(long execution_id,\r\n\t\t\t\t\t\t\t\t\t\tBulkRequest req, BulkResponse resp) {\r\n\t\t\t\t\t\t\t\t\t// TODO Auto-generated method stub\r\n\t\t\t\t\t\t\t\t\tSystem.out.println(\"execution id:\" + execution_id);\r\n\t\t\t\t\t\t\t\t\tSystem.out.println(req.toString());\r\n\t\t\t\t\t\t\t\t\tBulkItemResponse[] resp_items_arr = resp.getItems();\r\n\t\t\t\t\t\t\t\t\tfor(int i=0; i < resp_items_arr.length; i++){\r\n\t\t\t\t\t\t\t\t\t\tSystem.out.println(resp_items_arr[i].getItemId());\r\n\t\t\t\t\t\t\t\t\t}\r\n\t\t\t\t\t\t\t\t\tSystem.out.println(resp.hasFailures());\r\n\t\t\t\t\t\t\t\t}\r\n\r\n\t\t\t\t\t\t\t\t@Override\r\n\t\t\t\t\t\t\t\tpublic void afterBulk(long arg0,\r\n\t\t\t\t\t\t\t\t\t\tBulkRequest arg1, Throwable arg2) {\r\n\t\t\t\t\t\t\t\t\t// TODO Auto-generated method stub\r\n\t\t\t\t\t\t\t\t\tSystem.out.println(\"the whole bulk failed.--->\"\r\n\t\t\t\t\t\t\t\t\t\t\t+ arg2.getLocalizedMessage());\r\n\t\t\t\t\t\t\t\t}\r\n\r\n\t\t\t\t\t\t\t\t@Override\r\n\t\t\t\t\t\t\t\tpublic void beforeBulk(long arg0,\r\n\t\t\t\t\t\t\t\t\t\tBulkRequest arg1) {\r\n\t\t\t\t\t\t\t\t\t// TODO Auto-generated method stub\r\n\t\t\t\t\t\t\t\t\tSystem.out\r\n\t\t\t\t\t\t\t\t\t\t\t.println(\"these were the number of requests sent to es. bulk\");\r\n\t\t\t\t\t\t\t\t\tSystem.out.println(arg1.numberOfActions());\r\n\r\n\t\t\t\t\t\t\t\t}\r\n\r\n\t\t\t\t\t\t\t}).setBulkActions(10)\r\n\t\t\t\t\t.setBulkSize(new ByteSizeValue(1, ByteSizeUnit.GB))\r\n\t\t\t\t\t.setFlushInterval(TimeValue.timeValueSeconds(30))\r\n\t\t\t\t\t.setConcurrentRequests(1).build();\r\n\t\t}\r\n\t\treturn bulkprocessor;\r\n\t}\r\n        //**\r\n      \r\n\tpublic BulkRequestBuilder getBuilder() {\r\n\t\tif (this.builder == null) {\r\n                         //elasticsearchclientholder is a singleton holding a connection to ES.\r\n\t\t\tthis.builder = ElasticClientHolder.getInstance().prepareBulk();\r\n\t\t}\r\n\t\treturn builder;\r\n\t}\r\n\t\r\n\t\r\n\tpublic void build_and_test_bulk_request(){\r\n\t\tfor(int i=0; i < 10; i++){\r\n\t\t\tJSONObject elastic_object = new JSONObject();\r\n\t\t\telastic_object.put(\"test_name\", \"name_\" + String.valueOf(i));\r\n\t\t\tgetBulkprocessor().add(\r\n\t\t\t\t\tnew IndexRequest(\"test_index\",\r\n\t\t\t\t\t\t\t\"test_mapping\")\r\n\t\t\t\t\t\t\t.source(elastic_object.toJSONString()));\r\n\t\t}\r\n\t}\r\n\t\r\n\tpublic BulkProcessorTestClasses(){\r\n\t\tbuild_and_test_bulk_request();\r\n\t}\r\n\t\r\n        /***\r\n        * the maping contains a single string property called test_name.\r\n        * in an index called test_index, and a type test_mapping.\r\n        *\r\n        */\r\n\r\n      }\r\n\r\n\r\nCalling the build_and_test_bulk_requests returns nothing.\r\n\r\nRegards,\r\nRR.\r\n"},"comment":{"url":"https://api.github.com/repos/elastic/elasticsearch/issues/comments/96869915","html_url":"https://github.com/elastic/elasticsearch/issues/10839#issuecomment-96869915","issue_url":"https://api.github.com/repos/elastic/elasticsearch/issues/10839","id":96869915,"user":{"login":"rrphotosoft","id":4465221,"avatar_url":"https://avatars.githubusercontent.com/u/4465221?v=3","gravatar_id":"","url":"https://api.github.com/users/rrphotosoft","html_url":"https://github.com/rrphotosoft","followers_url":"https://api.github.com/users/rrphotosoft/followers","following_url":"https://api.github.com/users/rrphotosoft/following{/other_user}","gists_url":"https://api.github.com/users/rrphotosoft/gists{/gist_id}","starred_url":"https://api.github.com/users/rrphotosoft/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rrphotosoft/subscriptions","organizations_url":"https://api.github.com/users/rrphotosoft/orgs","repos_url":"https://api.github.com/users/rrphotosoft/repos","events_url":"https://api.github.com/users/rrphotosoft/events{/privacy}","received_events_url":"https://api.github.com/users/rrphotosoft/received_events","type":"User","site_admin":false},"created_at":"2015-04-28T01:19:20Z","updated_at":"2015-04-28T01:19:20Z","body":"Hi,\r\nI am trying to debug the issue.\r\nThe problem does not reside with flush or close.\r\nAs I mentioned in my initial post, the records are <i>being indexed</i>. So flush or close is not necessary, execute() is being correctly called.\r\nIt is the callback that is not triggered.\r\nI hope that helps to solve the issue.\r\nRegards,\r\nRR."}}