{"action":"created","comment":{"url":"https://api.github.com/repos/hazelcast/hazelcast/pulls/comments/38286803","id":38286803,"diff_hunk":"@@ -818,57 +817,57 @@ public boolean isEmpty() {\n         return idToKeys;\n     }\n \n-    protected void putAllInternal(final Map<? extends Object, ? extends Object> entries) {\n-        final NodeEngine nodeEngine = getNodeEngine();\n-        final MapService mapService = getService();\n-        int factor = 3;\n+    /**\n+     * This Operation will first group all puts per partition and then send a PutAllOperation per partition. So if there are e.g.\n+     * 5 keys for a single partition, then instead of having 5 remote invocations, there will be only 1 remote invocation.\n+     *\n+     * If there are multiple puts for different partitions on the same member, they are executed as different remote operations.\n+     * Probably this can be optimized in the future by making use of an PartitionIterating operation.\n+     *\n+     * @param entries\n+     */\n+    protected void putAllInternal(Map<? extends Object, ? extends Object> entries) {\n+        NodeEngine nodeEngine = getNodeEngine();\n         InternalPartitionService partitionService = nodeEngine.getPartitionService();\n         OperationService operationService = nodeEngine.getOperationService();\n         int partitionCount = partitionService.getPartitionCount();\n-        boolean tooManyEntries = entries.size() > (partitionCount * factor);\n+        MapServiceContext mapServiceContext = getService().getMapServiceContext();\n+\n         try {\n-            if (tooManyEntries) {\n-                List<Future> futures = new LinkedList<Future>();\n-                Map<Integer, MapEntrySet> entryMap\n-                        = new HashMap<Integer, MapEntrySet>(nodeEngine.getPartitionService().getPartitionCount());\n-                for (Entry entry : entries.entrySet()) {\n-                    checkNotNull(entry.getKey(), NULL_KEY_IS_NOT_ALLOWED);\n-                    checkNotNull(entry.getValue(), NULL_VALUE_IS_NOT_ALLOWED);\n-\n-                    int partitionId = partitionService.getPartitionId(entry.getKey());\n-                    if (!entryMap.containsKey(partitionId)) {\n-                        entryMap.put(partitionId, new MapEntrySet());\n-                    }\n-                    entryMap.get(partitionId).add(\n-                            new AbstractMap.SimpleImmutableEntry<Data, Data>(mapService.getMapServiceContext().toData(\n-                                    entry.getKey(),\n-                                    partitionStrategy),\n-                                    mapService.getMapServiceContext()\n-                                            .toData(entry.getValue())\n-                            ));\n+            List<Future> futures = new ArrayList<Future>(partitionCount);\n+            MapEntrySet[] entrySetPerPartition = new MapEntrySet[partitionCount];\n+\n+            // first we fill entrySetPerPartition\n+            for (Entry entry : entries.entrySet()) {\n+                checkNotNull(entry.getKey(), NULL_KEY_IS_NOT_ALLOWED);\n+                checkNotNull(entry.getValue(), NULL_VALUE_IS_NOT_ALLOWED);\n+\n+                int partitionId = partitionService.getPartitionId(entry.getKey());\n+                MapEntrySet entrySet = entrySetPerPartition[partitionId];\n+                if (entrySet == null) {\n+                    entrySet = new MapEntrySet();\n+                    entrySetPerPartition[partitionId] = entrySet;\n                 }\n \n-                for (final Map.Entry<Integer, MapEntrySet> entry : entryMap.entrySet()) {\n-                    final Integer partitionId = entry.getKey();\n-                    final PutAllOperation op = new PutAllOperation(name, entry.getValue());\n-                    op.setPartitionId(partitionId);\n-                    futures.add(operationService.invokeOnPartition(SERVICE_NAME, op, partitionId));\n-                }\n+                Data keyData = mapServiceContext.toData(entry.getKey(), partitionStrategy);\n+                Data valueData = mapServiceContext.toData(entry.getValue());\n+                entrySet.add(new AbstractMap.SimpleImmutableEntry<Data, Data>(keyData, valueData));\n+            }\n \n-                for (Future future : futures) {\n-                    future.get();\n+            // then we invoke the operations\n+            for (int partitionId = 0; partitionId < entrySetPerPartition.length; partitionId++) {\n+                MapEntrySet entrySet = entrySetPerPartition[partitionId];\n+                if (entrySet != null) {\n+                    // If there is a single entry, we could make use of a PutOperation since that is a bit cheaper","path":"hazelcast/src/main/java/com/hazelcast/map/impl/proxy/MapProxySupport.java","position":94,"original_position":94,"commit_id":"aa201a89ef4748ad69bb46c4a03aed42f6e5530c","original_commit_id":"aa201a89ef4748ad69bb46c4a03aed42f6e5530c","user":{"login":"pveentjer","id":105243,"avatar_url":"https://avatars.githubusercontent.com/u/105243?v=3","gravatar_id":"","url":"https://api.github.com/users/pveentjer","html_url":"https://github.com/pveentjer","followers_url":"https://api.github.com/users/pveentjer/followers","following_url":"https://api.github.com/users/pveentjer/following{/other_user}","gists_url":"https://api.github.com/users/pveentjer/gists{/gist_id}","starred_url":"https://api.github.com/users/pveentjer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pveentjer/subscriptions","organizations_url":"https://api.github.com/users/pveentjer/orgs","repos_url":"https://api.github.com/users/pveentjer/repos","events_url":"https://api.github.com/users/pveentjer/events{/privacy}","received_events_url":"https://api.github.com/users/pveentjer/received_events","type":"User","site_admin":false},"body":"Yes. \r\n\r\nWe should not work around a bug.. but we should fix the bug :)","created_at":"2015-08-31T06:21:27Z","updated_at":"2015-08-31T06:21:27Z","html_url":"https://github.com/hazelcast/hazelcast/pull/6094#discussion_r38286803","pull_request_url":"https://api.github.com/repos/hazelcast/hazelcast/pulls/6094","_links":{"self":{"href":"https://api.github.com/repos/hazelcast/hazelcast/pulls/comments/38286803"},"html":{"href":"https://github.com/hazelcast/hazelcast/pull/6094#discussion_r38286803"},"pull_request":{"href":"https://api.github.com/repos/hazelcast/hazelcast/pulls/6094"}}},"pull_request":{"url":"https://api.github.com/repos/hazelcast/hazelcast/pulls/6094","id":43732891,"html_url":"https://github.com/hazelcast/hazelcast/pull/6094","diff_url":"https://github.com/hazelcast/hazelcast/pull/6094.diff","patch_url":"https://github.com/hazelcast/hazelcast/pull/6094.patch","issue_url":"https://api.github.com/repos/hazelcast/hazelcast/issues/6094","number":6094,"state":"open","locked":false,"title":"Map.putAll always merges operation per partition","user":{"login":"pveentjer","id":105243,"avatar_url":"https://avatars.githubusercontent.com/u/105243?v=3","gravatar_id":"","url":"https://api.github.com/users/pveentjer","html_url":"https://github.com/pveentjer","followers_url":"https://api.github.com/users/pveentjer/followers","following_url":"https://api.github.com/users/pveentjer/following{/other_user}","gists_url":"https://api.github.com/users/pveentjer/gists{/gist_id}","starred_url":"https://api.github.com/users/pveentjer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pveentjer/subscriptions","organizations_url":"https://api.github.com/users/pveentjer/orgs","repos_url":"https://api.github.com/users/pveentjer/repos","events_url":"https://api.github.com/users/pveentjer/events{/privacy}","received_events_url":"https://api.github.com/users/pveentjer/received_events","type":"User","site_admin":false},"body":"There was some strange and undocumented logic that only above a certain\r\nthreshold the put operations were being merged per partition. This PR fixes\r\nthat by always doing so.\r\n\r\nEven if there are 2 entries, but they both fall in the same partition, it is already very beneficial to merge these operations. There is really no reason for a threshold. \r\n\r\nIn the future we can speed it up even more by making use of the PartitionIteratingOperation so that multiple partitions for the same member, can be merged into a single remote call. But this needs to be investigated.","created_at":"2015-08-31T05:25:28Z","updated_at":"2015-08-31T06:21:27Z","closed_at":null,"merged_at":null,"merge_commit_sha":"009c076305c69387f8ea74cdcbaaef95ec5354fb","assignee":null,"milestone":{"url":"https://api.github.com/repos/hazelcast/hazelcast/milestones/61","html_url":"https://github.com/hazelcast/hazelcast/milestones/3.6","labels_url":"https://api.github.com/repos/hazelcast/hazelcast/milestones/61/labels","id":982584,"number":61,"title":"3.6","description":"","creator":{"login":"enesakar","id":234086,"avatar_url":"https://avatars.githubusercontent.com/u/234086?v=3","gravatar_id":"","url":"https://api.github.com/users/enesakar","html_url":"https://github.com/enesakar","followers_url":"https://api.github.com/users/enesakar/followers","following_url":"https://api.github.com/users/enesakar/following{/other_user}","gists_url":"https://api.github.com/users/enesakar/gists{/gist_id}","starred_url":"https://api.github.com/users/enesakar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/enesakar/subscriptions","organizations_url":"https://api.github.com/users/enesakar/orgs","repos_url":"https://api.github.com/users/enesakar/repos","events_url":"https://api.github.com/users/enesakar/events{/privacy}","received_events_url":"https://api.github.com/users/enesakar/received_events","type":"User","site_admin":false},"open_issues":147,"closed_issues":331,"state":"open","created_at":"2015-02-17T09:27:09Z","updated_at":"2015-08-31T05:25:28Z","due_on":null,"closed_at":null},"commits_url":"https://api.github.com/repos/hazelcast/hazelcast/pulls/6094/commits","review_comments_url":"https://api.github.com/repos/hazelcast/hazelcast/pulls/6094/comments","review_comment_url":"https://api.github.com/repos/hazelcast/hazelcast/pulls/comments{/number}","comments_url":"https://api.github.com/repos/hazelcast/hazelcast/issues/6094/comments","statuses_url":"https://api.github.com/repos/hazelcast/hazelcast/statuses/aa201a89ef4748ad69bb46c4a03aed42f6e5530c","head":{"label":"pveentjer:performance/3.x/map-put-all","ref":"performance/3.x/map-put-all","sha":"aa201a89ef4748ad69bb46c4a03aed42f6e5530c","user":{"login":"pveentjer","id":105243,"avatar_url":"https://avatars.githubusercontent.com/u/105243?v=3","gravatar_id":"","url":"https://api.github.com/users/pveentjer","html_url":"https://github.com/pveentjer","followers_url":"https://api.github.com/users/pveentjer/followers","following_url":"https://api.github.com/users/pveentjer/following{/other_user}","gists_url":"https://api.github.com/users/pveentjer/gists{/gist_id}","starred_url":"https://api.github.com/users/pveentjer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pveentjer/subscriptions","organizations_url":"https://api.github.com/users/pveentjer/orgs","repos_url":"https://api.github.com/users/pveentjer/repos","events_url":"https://api.github.com/users/pveentjer/events{/privacy}","received_events_url":"https://api.github.com/users/pveentjer/received_events","type":"User","site_admin":false},"repo":{"id":10990111,"name":"hazelcast","full_name":"pveentjer/hazelcast","owner":{"login":"pveentjer","id":105243,"avatar_url":"https://avatars.githubusercontent.com/u/105243?v=3","gravatar_id":"","url":"https://api.github.com/users/pveentjer","html_url":"https://github.com/pveentjer","followers_url":"https://api.github.com/users/pveentjer/followers","following_url":"https://api.github.com/users/pveentjer/following{/other_user}","gists_url":"https://api.github.com/users/pveentjer/gists{/gist_id}","starred_url":"https://api.github.com/users/pveentjer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pveentjer/subscriptions","organizations_url":"https://api.github.com/users/pveentjer/orgs","repos_url":"https://api.github.com/users/pveentjer/repos","events_url":"https://api.github.com/users/pveentjer/events{/privacy}","received_events_url":"https://api.github.com/users/pveentjer/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/pveentjer/hazelcast","description":"Open Source In-Memory Data Grid","fork":true,"url":"https://api.github.com/repos/pveentjer/hazelcast","forks_url":"https://api.github.com/repos/pveentjer/hazelcast/forks","keys_url":"https://api.github.com/repos/pveentjer/hazelcast/keys{/key_id}","collaborators_url":"https://api.github.com/repos/pveentjer/hazelcast/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/pveentjer/hazelcast/teams","hooks_url":"https://api.github.com/repos/pveentjer/hazelcast/hooks","issue_events_url":"https://api.github.com/repos/pveentjer/hazelcast/issues/events{/number}","events_url":"https://api.github.com/repos/pveentjer/hazelcast/events","assignees_url":"https://api.github.com/repos/pveentjer/hazelcast/assignees{/user}","branches_url":"https://api.github.com/repos/pveentjer/hazelcast/branches{/branch}","tags_url":"https://api.github.com/repos/pveentjer/hazelcast/tags","blobs_url":"https://api.github.com/repos/pveentjer/hazelcast/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/pveentjer/hazelcast/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/pveentjer/hazelcast/git/refs{/sha}","trees_url":"https://api.github.com/repos/pveentjer/hazelcast/git/trees{/sha}","statuses_url":"https://api.github.com/repos/pveentjer/hazelcast/statuses/{sha}","languages_url":"https://api.github.com/repos/pveentjer/hazelcast/languages","stargazers_url":"https://api.github.com/repos/pveentjer/hazelcast/stargazers","contributors_url":"https://api.github.com/repos/pveentjer/hazelcast/contributors","subscribers_url":"https://api.github.com/repos/pveentjer/hazelcast/subscribers","subscription_url":"https://api.github.com/repos/pveentjer/hazelcast/subscription","commits_url":"https://api.github.com/repos/pveentjer/hazelcast/commits{/sha}","git_commits_url":"https://api.github.com/repos/pveentjer/hazelcast/git/commits{/sha}","comments_url":"https://api.github.com/repos/pveentjer/hazelcast/comments{/number}","issue_comment_url":"https://api.github.com/repos/pveentjer/hazelcast/issues/comments{/number}","contents_url":"https://api.github.com/repos/pveentjer/hazelcast/contents/{+path}","compare_url":"https://api.github.com/repos/pveentjer/hazelcast/compare/{base}...{head}","merges_url":"https://api.github.com/repos/pveentjer/hazelcast/merges","archive_url":"https://api.github.com/repos/pveentjer/hazelcast/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/pveentjer/hazelcast/downloads","issues_url":"https://api.github.com/repos/pveentjer/hazelcast/issues{/number}","pulls_url":"https://api.github.com/repos/pveentjer/hazelcast/pulls{/number}","milestones_url":"https://api.github.com/repos/pveentjer/hazelcast/milestones{/number}","notifications_url":"https://api.github.com/repos/pveentjer/hazelcast/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/pveentjer/hazelcast/labels{/name}","releases_url":"https://api.github.com/repos/pveentjer/hazelcast/releases{/id}","created_at":"2013-06-27T07:15:08Z","updated_at":"2015-04-22T04:00:47Z","pushed_at":"2015-08-31T05:29:20Z","git_url":"git://github.com/pveentjer/hazelcast.git","ssh_url":"git@github.com:pveentjer/hazelcast.git","clone_url":"https://github.com/pveentjer/hazelcast.git","svn_url":"https://github.com/pveentjer/hazelcast","homepage":"www.hazelcast.com","size":80702,"stargazers_count":0,"watchers_count":0,"language":"Java","has_issues":false,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":1,"mirror_url":null,"open_issues_count":0,"forks":1,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"hazelcast:master","ref":"master","sha":"410d077538e8af21fc7846a758d83cc67801da83","user":{"login":"hazelcast","id":1453152,"avatar_url":"https://avatars.githubusercontent.com/u/1453152?v=3","gravatar_id":"","url":"https://api.github.com/users/hazelcast","html_url":"https://github.com/hazelcast","followers_url":"https://api.github.com/users/hazelcast/followers","following_url":"https://api.github.com/users/hazelcast/following{/other_user}","gists_url":"https://api.github.com/users/hazelcast/gists{/gist_id}","starred_url":"https://api.github.com/users/hazelcast/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hazelcast/subscriptions","organizations_url":"https://api.github.com/users/hazelcast/orgs","repos_url":"https://api.github.com/users/hazelcast/repos","events_url":"https://api.github.com/users/hazelcast/events{/privacy}","received_events_url":"https://api.github.com/users/hazelcast/received_events","type":"Organization","site_admin":false},"repo":{"id":3786237,"name":"hazelcast","full_name":"hazelcast/hazelcast","owner":{"login":"hazelcast","id":1453152,"avatar_url":"https://avatars.githubusercontent.com/u/1453152?v=3","gravatar_id":"","url":"https://api.github.com/users/hazelcast","html_url":"https://github.com/hazelcast","followers_url":"https://api.github.com/users/hazelcast/followers","following_url":"https://api.github.com/users/hazelcast/following{/other_user}","gists_url":"https://api.github.com/users/hazelcast/gists{/gist_id}","starred_url":"https://api.github.com/users/hazelcast/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hazelcast/subscriptions","organizations_url":"https://api.github.com/users/hazelcast/orgs","repos_url":"https://api.github.com/users/hazelcast/repos","events_url":"https://api.github.com/users/hazelcast/events{/privacy}","received_events_url":"https://api.github.com/users/hazelcast/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/hazelcast/hazelcast","description":"Open Source In-Memory Data Grid","fork":false,"url":"https://api.github.com/repos/hazelcast/hazelcast","forks_url":"https://api.github.com/repos/hazelcast/hazelcast/forks","keys_url":"https://api.github.com/repos/hazelcast/hazelcast/keys{/key_id}","collaborators_url":"https://api.github.com/repos/hazelcast/hazelcast/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/hazelcast/hazelcast/teams","hooks_url":"https://api.github.com/repos/hazelcast/hazelcast/hooks","issue_events_url":"https://api.github.com/repos/hazelcast/hazelcast/issues/events{/number}","events_url":"https://api.github.com/repos/hazelcast/hazelcast/events","assignees_url":"https://api.github.com/repos/hazelcast/hazelcast/assignees{/user}","branches_url":"https://api.github.com/repos/hazelcast/hazelcast/branches{/branch}","tags_url":"https://api.github.com/repos/hazelcast/hazelcast/tags","blobs_url":"https://api.github.com/repos/hazelcast/hazelcast/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/hazelcast/hazelcast/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/hazelcast/hazelcast/git/refs{/sha}","trees_url":"https://api.github.com/repos/hazelcast/hazelcast/git/trees{/sha}","statuses_url":"https://api.github.com/repos/hazelcast/hazelcast/statuses/{sha}","languages_url":"https://api.github.com/repos/hazelcast/hazelcast/languages","stargazers_url":"https://api.github.com/repos/hazelcast/hazelcast/stargazers","contributors_url":"https://api.github.com/repos/hazelcast/hazelcast/contributors","subscribers_url":"https://api.github.com/repos/hazelcast/hazelcast/subscribers","subscription_url":"https://api.github.com/repos/hazelcast/hazelcast/subscription","commits_url":"https://api.github.com/repos/hazelcast/hazelcast/commits{/sha}","git_commits_url":"https://api.github.com/repos/hazelcast/hazelcast/git/commits{/sha}","comments_url":"https://api.github.com/repos/hazelcast/hazelcast/comments{/number}","issue_comment_url":"https://api.github.com/repos/hazelcast/hazelcast/issues/comments{/number}","contents_url":"https://api.github.com/repos/hazelcast/hazelcast/contents/{+path}","compare_url":"https://api.github.com/repos/hazelcast/hazelcast/compare/{base}...{head}","merges_url":"https://api.github.com/repos/hazelcast/hazelcast/merges","archive_url":"https://api.github.com/repos/hazelcast/hazelcast/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/hazelcast/hazelcast/downloads","issues_url":"https://api.github.com/repos/hazelcast/hazelcast/issues{/number}","pulls_url":"https://api.github.com/repos/hazelcast/hazelcast/pulls{/number}","milestones_url":"https://api.github.com/repos/hazelcast/hazelcast/milestones{/number}","notifications_url":"https://api.github.com/repos/hazelcast/hazelcast/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/hazelcast/hazelcast/labels{/name}","releases_url":"https://api.github.com/repos/hazelcast/hazelcast/releases{/id}","created_at":"2012-03-21T12:25:18Z","updated_at":"2015-08-28T21:25:03Z","pushed_at":"2015-08-31T05:29:20Z","git_url":"git://github.com/hazelcast/hazelcast.git","ssh_url":"git@github.com:hazelcast/hazelcast.git","clone_url":"https://github.com/hazelcast/hazelcast.git","svn_url":"https://github.com/hazelcast/hazelcast","homepage":"www.hazelcast.com","size":151189,"stargazers_count":1220,"watchers_count":1220,"language":"Java","has_issues":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":590,"mirror_url":null,"open_issues_count":497,"forks":590,"open_issues":497,"watchers":1220,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/hazelcast/hazelcast/pulls/6094"},"html":{"href":"https://github.com/hazelcast/hazelcast/pull/6094"},"issue":{"href":"https://api.github.com/repos/hazelcast/hazelcast/issues/6094"},"comments":{"href":"https://api.github.com/repos/hazelcast/hazelcast/issues/6094/comments"},"review_comments":{"href":"https://api.github.com/repos/hazelcast/hazelcast/pulls/6094/comments"},"review_comment":{"href":"https://api.github.com/repos/hazelcast/hazelcast/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/hazelcast/hazelcast/pulls/6094/commits"},"statuses":{"href":"https://api.github.com/repos/hazelcast/hazelcast/statuses/aa201a89ef4748ad69bb46c4a03aed42f6e5530c"}}}}