{"action":"created","comment":{"url":"https://api.github.com/repos/facebook/presto/pulls/comments/45536302","id":45536302,"diff_hunk":"@@ -66,6 +79,98 @@\n \n     private HiveBucketing() {}\n \n+    public static int getHiveBucket(List<TypeInfo> types, List<Block> blocks, int position, int bucketCount)\n+    {\n+        return (getBucketHashCode(types, blocks, position) & Integer.MAX_VALUE) % bucketCount;\n+    }\n+\n+    public static int getBucketHashCode(List<TypeInfo> types, List<Block> blocks, int position)\n+    {\n+        int result = 0;\n+        for (int i = 0; i < blocks.size(); i++) {\n+            int fieldHash = hash(types.get(i), blocks.get(i), position);\n+            result = result * 31 + fieldHash;\n+        }\n+        return result;\n+    }\n+\n+    private static int hash(TypeInfo type, Block block, int position)\n+    {\n+        // This function mirrors the behavior of function hashCode in\n+        // HIVE-12025 ba83fd7bff serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorUtils.java\n+\n+        // HIVE-7148 proposed change to bucketing hash algorithms. If that gets implemented, this function will need to change significantly.\n+\n+        if (block.isNull(position)) {\n+            return 0;\n+        }\n+\n+        switch (type.getCategory()) {\n+            case PRIMITIVE:\n+                PrimitiveCategory primitiveCategory = ((PrimitiveTypeInfo) type).getPrimitiveCategory();\n+                switch (primitiveCategory) {\n+                    case BOOLEAN:\n+                        return BOOLEAN.getBoolean(block, position) ? 1 : 0;\n+                    case BYTE:\n+                    case SHORT:\n+                    case INT:\n+                        return Ints.checkedCast(BIGINT.getLong(block, position));\n+                    case LONG:\n+                        long bigintValue = BIGINT.getLong(block, position);\n+                        return (int) ((bigintValue >>> 32) ^ bigintValue);\n+                    case FLOAT:\n+                        return Float.floatToIntBits((float) DOUBLE.getDouble(block, position));\n+                    case DOUBLE:\n+                        long doubleValue = doubleToLongBits(DOUBLE.getDouble(block, position));\n+                        return (int) ((doubleValue >>> 32) ^ doubleValue);\n+                    case STRING:\n+                        return hashBytes(0, VARCHAR.getSlice(block, position));\n+                    case DATE:\n+                        // day offset from 1970-01-01\n+                        long days = DATE.getLong(block, position);\n+                        return Ints.checkedCast(days);\n+                    case TIMESTAMP:\n+                        long millisSinceEpoch = TIMESTAMP.getLong(block, position);\n+                        // seconds << 30 + nanoseconds\n+                        long secondsAndNanos = (Math.floorDiv(millisSinceEpoch, 1000L) << 30) + Math.floorMod(millisSinceEpoch, 1000);\n+                        return (int) ((secondsAndNanos >>> 32) ^ secondsAndNanos);\n+                    default:\n+                        throw new UnsupportedOperationException(\"Computation of Hive bucket hashCode is not supported for Hive primitive category: \" + primitiveCategory.toString() + \".\");\n+                }\n+            case LIST: {\n+                TypeInfo elementTypeInfo = ((ListTypeInfo) type).getListElementTypeInfo();\n+                Block elementsBlock = block.getObject(position, Block.class);\n+                int result = 0;\n+                for (int i = 0; i < elementsBlock.getPositionCount(); i++) {\n+                    result = result * 31 + hash(elementTypeInfo, elementsBlock, i);\n+                }\n+                return result;\n+            }\n+            case MAP: {\n+                MapTypeInfo mapTypeInfo = (MapTypeInfo) type;\n+                TypeInfo keyTypeInfo = mapTypeInfo.getMapKeyTypeInfo();\n+                TypeInfo valueTypeInfo = mapTypeInfo.getMapValueTypeInfo();\n+                Block elementsBlock = block.getObject(position, Block.class);\n+                int result = 0;\n+                for (int i = 0; i < elementsBlock.getPositionCount(); i += 2) {\n+                    result += hash(keyTypeInfo, elementsBlock, i) ^ hash(valueTypeInfo, elementsBlock, i + 1);\n+                }\n+                return result;\n+            }\n+            default:\n+                throw new UnsupportedOperationException(\"Computation of Hive bucket hashCode is not supported for Hive category: \" + type.getCategory().toString() + \".\");\n+        }\n+    }\n+\n+    private static int hashBytes(int initialValue, Slice bytes)\n+    {\n+        int result = initialValue;\n+        for (byte b : bytes.getBytes()) {","path":"presto-hive/src/main/java/com/facebook/presto/hive/HiveBucketing.java","position":129,"original_position":129,"commit_id":"4f0cc97feedad9591f510c0b4a2074cfae19ef6f","original_commit_id":"4f0cc97feedad9591f510c0b4a2074cfae19ef6f","user":{"login":"dain","id":58298,"avatar_url":"https://avatars.githubusercontent.com/u/58298?v=3","gravatar_id":"","url":"https://api.github.com/users/dain","html_url":"https://github.com/dain","followers_url":"https://api.github.com/users/dain/followers","following_url":"https://api.github.com/users/dain/following{/other_user}","gists_url":"https://api.github.com/users/dain/gists{/gist_id}","starred_url":"https://api.github.com/users/dain/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dain/subscriptions","organizations_url":"https://api.github.com/users/dain/orgs","repos_url":"https://api.github.com/users/dain/repos","events_url":"https://api.github.com/users/dain/events{/privacy}","received_events_url":"https://api.github.com/users/dain/received_events","type":"User","site_admin":false},"body":"Write this as \r\n```java\r\nfor (int i = 0; i < bytes.length(); i++) {\r\n    result = result * 31 + bytes.getByte(i);\r\n}\r\n```\r\nThe call to `getBytes` creates a byte array and copies the data into the new byte array.","created_at":"2015-11-21T00:36:59Z","updated_at":"2015-11-21T00:36:59Z","html_url":"https://github.com/facebook/presto/pull/4009#discussion_r45536302","pull_request_url":"https://api.github.com/repos/facebook/presto/pulls/4009","_links":{"self":{"href":"https://api.github.com/repos/facebook/presto/pulls/comments/45536302"},"html":{"href":"https://github.com/facebook/presto/pull/4009#discussion_r45536302"},"pull_request":{"href":"https://api.github.com/repos/facebook/presto/pulls/4009"}}},"pull_request":{"url":"https://api.github.com/repos/facebook/presto/pulls/4009","id":51413063,"html_url":"https://github.com/facebook/presto/pull/4009","diff_url":"https://github.com/facebook/presto/pull/4009.diff","patch_url":"https://github.com/facebook/presto/pull/4009.patch","issue_url":"https://api.github.com/repos/facebook/presto/issues/4009","number":4009,"state":"open","locked":false,"title":"Add getHiveBucket that takes Block and position","user":{"login":"haozhun","id":2020503,"avatar_url":"https://avatars.githubusercontent.com/u/2020503?v=3","gravatar_id":"","url":"https://api.github.com/users/haozhun","html_url":"https://github.com/haozhun","followers_url":"https://api.github.com/users/haozhun/followers","following_url":"https://api.github.com/users/haozhun/following{/other_user}","gists_url":"https://api.github.com/users/haozhun/gists{/gist_id}","starred_url":"https://api.github.com/users/haozhun/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/haozhun/subscriptions","organizations_url":"https://api.github.com/users/haozhun/orgs","repos_url":"https://api.github.com/users/haozhun/repos","events_url":"https://api.github.com/users/haozhun/events{/privacy}","received_events_url":"https://api.github.com/users/haozhun/received_events","type":"User","site_admin":false},"body":"","created_at":"2015-11-20T21:52:17Z","updated_at":"2015-11-21T00:36:59Z","closed_at":null,"merged_at":null,"merge_commit_sha":"6b2d7df1245331bc0b2ce8f44449dd486164661b","assignee":{"login":"dain","id":58298,"avatar_url":"https://avatars.githubusercontent.com/u/58298?v=3","gravatar_id":"","url":"https://api.github.com/users/dain","html_url":"https://github.com/dain","followers_url":"https://api.github.com/users/dain/followers","following_url":"https://api.github.com/users/dain/following{/other_user}","gists_url":"https://api.github.com/users/dain/gists{/gist_id}","starred_url":"https://api.github.com/users/dain/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dain/subscriptions","organizations_url":"https://api.github.com/users/dain/orgs","repos_url":"https://api.github.com/users/dain/repos","events_url":"https://api.github.com/users/dain/events{/privacy}","received_events_url":"https://api.github.com/users/dain/received_events","type":"User","site_admin":false},"milestone":null,"commits_url":"https://api.github.com/repos/facebook/presto/pulls/4009/commits","review_comments_url":"https://api.github.com/repos/facebook/presto/pulls/4009/comments","review_comment_url":"https://api.github.com/repos/facebook/presto/pulls/comments{/number}","comments_url":"https://api.github.com/repos/facebook/presto/issues/4009/comments","statuses_url":"https://api.github.com/repos/facebook/presto/statuses/4f0cc97feedad9591f510c0b4a2074cfae19ef6f","head":{"label":"haozhun:hash","ref":"hash","sha":"4f0cc97feedad9591f510c0b4a2074cfae19ef6f","user":{"login":"haozhun","id":2020503,"avatar_url":"https://avatars.githubusercontent.com/u/2020503?v=3","gravatar_id":"","url":"https://api.github.com/users/haozhun","html_url":"https://github.com/haozhun","followers_url":"https://api.github.com/users/haozhun/followers","following_url":"https://api.github.com/users/haozhun/following{/other_user}","gists_url":"https://api.github.com/users/haozhun/gists{/gist_id}","starred_url":"https://api.github.com/users/haozhun/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/haozhun/subscriptions","organizations_url":"https://api.github.com/users/haozhun/orgs","repos_url":"https://api.github.com/users/haozhun/repos","events_url":"https://api.github.com/users/haozhun/events{/privacy}","received_events_url":"https://api.github.com/users/haozhun/received_events","type":"User","site_admin":false},"repo":{"id":31032158,"name":"presto","full_name":"haozhun/presto","owner":{"login":"haozhun","id":2020503,"avatar_url":"https://avatars.githubusercontent.com/u/2020503?v=3","gravatar_id":"","url":"https://api.github.com/users/haozhun","html_url":"https://github.com/haozhun","followers_url":"https://api.github.com/users/haozhun/followers","following_url":"https://api.github.com/users/haozhun/following{/other_user}","gists_url":"https://api.github.com/users/haozhun/gists{/gist_id}","starred_url":"https://api.github.com/users/haozhun/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/haozhun/subscriptions","organizations_url":"https://api.github.com/users/haozhun/orgs","repos_url":"https://api.github.com/users/haozhun/repos","events_url":"https://api.github.com/users/haozhun/events{/privacy}","received_events_url":"https://api.github.com/users/haozhun/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/haozhun/presto","description":"Distributed SQL query engine for running interactive analytic queries against big data sources.","fork":true,"url":"https://api.github.com/repos/haozhun/presto","forks_url":"https://api.github.com/repos/haozhun/presto/forks","keys_url":"https://api.github.com/repos/haozhun/presto/keys{/key_id}","collaborators_url":"https://api.github.com/repos/haozhun/presto/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/haozhun/presto/teams","hooks_url":"https://api.github.com/repos/haozhun/presto/hooks","issue_events_url":"https://api.github.com/repos/haozhun/presto/issues/events{/number}","events_url":"https://api.github.com/repos/haozhun/presto/events","assignees_url":"https://api.github.com/repos/haozhun/presto/assignees{/user}","branches_url":"https://api.github.com/repos/haozhun/presto/branches{/branch}","tags_url":"https://api.github.com/repos/haozhun/presto/tags","blobs_url":"https://api.github.com/repos/haozhun/presto/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/haozhun/presto/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/haozhun/presto/git/refs{/sha}","trees_url":"https://api.github.com/repos/haozhun/presto/git/trees{/sha}","statuses_url":"https://api.github.com/repos/haozhun/presto/statuses/{sha}","languages_url":"https://api.github.com/repos/haozhun/presto/languages","stargazers_url":"https://api.github.com/repos/haozhun/presto/stargazers","contributors_url":"https://api.github.com/repos/haozhun/presto/contributors","subscribers_url":"https://api.github.com/repos/haozhun/presto/subscribers","subscription_url":"https://api.github.com/repos/haozhun/presto/subscription","commits_url":"https://api.github.com/repos/haozhun/presto/commits{/sha}","git_commits_url":"https://api.github.com/repos/haozhun/presto/git/commits{/sha}","comments_url":"https://api.github.com/repos/haozhun/presto/comments{/number}","issue_comment_url":"https://api.github.com/repos/haozhun/presto/issues/comments{/number}","contents_url":"https://api.github.com/repos/haozhun/presto/contents/{+path}","compare_url":"https://api.github.com/repos/haozhun/presto/compare/{base}...{head}","merges_url":"https://api.github.com/repos/haozhun/presto/merges","archive_url":"https://api.github.com/repos/haozhun/presto/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/haozhun/presto/downloads","issues_url":"https://api.github.com/repos/haozhun/presto/issues{/number}","pulls_url":"https://api.github.com/repos/haozhun/presto/pulls{/number}","milestones_url":"https://api.github.com/repos/haozhun/presto/milestones{/number}","notifications_url":"https://api.github.com/repos/haozhun/presto/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/haozhun/presto/labels{/name}","releases_url":"https://api.github.com/repos/haozhun/presto/releases{/id}","created_at":"2015-02-19T19:19:31Z","updated_at":"2015-04-04T00:15:06Z","pushed_at":"2015-11-20T23:58:55Z","git_url":"git://github.com/haozhun/presto.git","ssh_url":"git@github.com:haozhun/presto.git","clone_url":"https://github.com/haozhun/presto.git","svn_url":"https://github.com/haozhun/presto","homepage":"https://prestodb.io","size":48838,"stargazers_count":0,"watchers_count":0,"language":"Java","has_issues":false,"has_downloads":true,"has_wiki":false,"has_pages":true,"forks_count":1,"mirror_url":null,"open_issues_count":0,"forks":1,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"facebook:master","ref":"master","sha":"54b476096dd69bc6f8c8841084206bc0bceb5db8","user":{"login":"facebook","id":69631,"avatar_url":"https://avatars.githubusercontent.com/u/69631?v=3","gravatar_id":"","url":"https://api.github.com/users/facebook","html_url":"https://github.com/facebook","followers_url":"https://api.github.com/users/facebook/followers","following_url":"https://api.github.com/users/facebook/following{/other_user}","gists_url":"https://api.github.com/users/facebook/gists{/gist_id}","starred_url":"https://api.github.com/users/facebook/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/facebook/subscriptions","organizations_url":"https://api.github.com/users/facebook/orgs","repos_url":"https://api.github.com/users/facebook/repos","events_url":"https://api.github.com/users/facebook/events{/privacy}","received_events_url":"https://api.github.com/users/facebook/received_events","type":"Organization","site_admin":false},"repo":{"id":5349565,"name":"presto","full_name":"facebook/presto","owner":{"login":"facebook","id":69631,"avatar_url":"https://avatars.githubusercontent.com/u/69631?v=3","gravatar_id":"","url":"https://api.github.com/users/facebook","html_url":"https://github.com/facebook","followers_url":"https://api.github.com/users/facebook/followers","following_url":"https://api.github.com/users/facebook/following{/other_user}","gists_url":"https://api.github.com/users/facebook/gists{/gist_id}","starred_url":"https://api.github.com/users/facebook/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/facebook/subscriptions","organizations_url":"https://api.github.com/users/facebook/orgs","repos_url":"https://api.github.com/users/facebook/repos","events_url":"https://api.github.com/users/facebook/events{/privacy}","received_events_url":"https://api.github.com/users/facebook/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/facebook/presto","description":"Distributed SQL query engine for running interactive analytic queries against big data sources.","fork":false,"url":"https://api.github.com/repos/facebook/presto","forks_url":"https://api.github.com/repos/facebook/presto/forks","keys_url":"https://api.github.com/repos/facebook/presto/keys{/key_id}","collaborators_url":"https://api.github.com/repos/facebook/presto/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/facebook/presto/teams","hooks_url":"https://api.github.com/repos/facebook/presto/hooks","issue_events_url":"https://api.github.com/repos/facebook/presto/issues/events{/number}","events_url":"https://api.github.com/repos/facebook/presto/events","assignees_url":"https://api.github.com/repos/facebook/presto/assignees{/user}","branches_url":"https://api.github.com/repos/facebook/presto/branches{/branch}","tags_url":"https://api.github.com/repos/facebook/presto/tags","blobs_url":"https://api.github.com/repos/facebook/presto/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/facebook/presto/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/facebook/presto/git/refs{/sha}","trees_url":"https://api.github.com/repos/facebook/presto/git/trees{/sha}","statuses_url":"https://api.github.com/repos/facebook/presto/statuses/{sha}","languages_url":"https://api.github.com/repos/facebook/presto/languages","stargazers_url":"https://api.github.com/repos/facebook/presto/stargazers","contributors_url":"https://api.github.com/repos/facebook/presto/contributors","subscribers_url":"https://api.github.com/repos/facebook/presto/subscribers","subscription_url":"https://api.github.com/repos/facebook/presto/subscription","commits_url":"https://api.github.com/repos/facebook/presto/commits{/sha}","git_commits_url":"https://api.github.com/repos/facebook/presto/git/commits{/sha}","comments_url":"https://api.github.com/repos/facebook/presto/comments{/number}","issue_comment_url":"https://api.github.com/repos/facebook/presto/issues/comments{/number}","contents_url":"https://api.github.com/repos/facebook/presto/contents/{+path}","compare_url":"https://api.github.com/repos/facebook/presto/compare/{base}...{head}","merges_url":"https://api.github.com/repos/facebook/presto/merges","archive_url":"https://api.github.com/repos/facebook/presto/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/facebook/presto/downloads","issues_url":"https://api.github.com/repos/facebook/presto/issues{/number}","pulls_url":"https://api.github.com/repos/facebook/presto/pulls{/number}","milestones_url":"https://api.github.com/repos/facebook/presto/milestones{/number}","notifications_url":"https://api.github.com/repos/facebook/presto/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/facebook/presto/labels{/name}","releases_url":"https://api.github.com/repos/facebook/presto/releases{/id}","created_at":"2012-08-09T01:03:37Z","updated_at":"2015-11-20T21:49:49Z","pushed_at":"2015-11-21T00:16:37Z","git_url":"git://github.com/facebook/presto.git","ssh_url":"git@github.com:facebook/presto.git","clone_url":"https://github.com/facebook/presto.git","svn_url":"https://github.com/facebook/presto","homepage":"https://prestodb.io","size":102725,"stargazers_count":4128,"watchers_count":4128,"language":"Java","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":true,"forks_count":1087,"mirror_url":null,"open_issues_count":376,"forks":1087,"open_issues":376,"watchers":4128,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/facebook/presto/pulls/4009"},"html":{"href":"https://github.com/facebook/presto/pull/4009"},"issue":{"href":"https://api.github.com/repos/facebook/presto/issues/4009"},"comments":{"href":"https://api.github.com/repos/facebook/presto/issues/4009/comments"},"review_comments":{"href":"https://api.github.com/repos/facebook/presto/pulls/4009/comments"},"review_comment":{"href":"https://api.github.com/repos/facebook/presto/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/facebook/presto/pulls/4009/commits"},"statuses":{"href":"https://api.github.com/repos/facebook/presto/statuses/4f0cc97feedad9591f510c0b4a2074cfae19ef6f"}}}}