{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/7945","labels_url":"https://api.github.com/repos/apache/spark/issues/7945/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/7945/comments","events_url":"https://api.github.com/repos/apache/spark/issues/7945/events","html_url":"https://github.com/apache/spark/pull/7945","id":99069950,"number":7945,"title":"[SPARK-9615][SPARK-9616][SQL][MLLIB] Bugs related to FrequentItems when merging and with Tungsten","user":{"login":"brkyvz","id":5243515,"avatar_url":"https://avatars.githubusercontent.com/u/5243515?v=3","gravatar_id":"","url":"https://api.github.com/users/brkyvz","html_url":"https://github.com/brkyvz","followers_url":"https://api.github.com/users/brkyvz/followers","following_url":"https://api.github.com/users/brkyvz/following{/other_user}","gists_url":"https://api.github.com/users/brkyvz/gists{/gist_id}","starred_url":"https://api.github.com/users/brkyvz/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/brkyvz/subscriptions","organizations_url":"https://api.github.com/users/brkyvz/orgs","repos_url":"https://api.github.com/users/brkyvz/repos","events_url":"https://api.github.com/users/brkyvz/events{/privacy}","received_events_url":"https://api.github.com/users/brkyvz/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2015-08-04T21:52:40Z","updated_at":"2015-08-04T21:52:57Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/7945","html_url":"https://github.com/apache/spark/pull/7945","diff_url":"https://github.com/apache/spark/pull/7945.diff","patch_url":"https://github.com/apache/spark/pull/7945.patch"},"body":"In short:\r\n1- FrequentItems should not use the InternalRow representation, because the keys in the map get messed up. For example, every key in the Map correspond to the very last element observed in the partition, when the elements are strings.\r\n\r\n2- Merging two partitions had a bug:\r\n\r\n**Existing behavior with size 3**\r\nPartition A -> Map(1 -> 3, 2 -> 3, 3 -> 4)\r\nPartition B -> Map(4 -> 25)\r\nResult -> Map()\r\n\r\n**Correct Behavior:**\r\nPartition A -> Map(1 -> 3, 2 -> 3, 3 -> 4)\r\nPartition B -> Map(4 -> 25)\r\nResult -> Map(3 -> 1, 4 -> 22)\r\n\r\ncc @mengxr @rxin @JoshRosen \r\n"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/127772877","html_url":"https://github.com/apache/spark/pull/7945#issuecomment-127772877","issue_url":"https://api.github.com/repos/apache/spark/issues/7945","id":127772877,"user":{"login":"AmplabJenkins","id":4031291,"avatar_url":"https://avatars.githubusercontent.com/u/4031291?v=3","gravatar_id":"","url":"https://api.github.com/users/AmplabJenkins","html_url":"https://github.com/AmplabJenkins","followers_url":"https://api.github.com/users/AmplabJenkins/followers","following_url":"https://api.github.com/users/AmplabJenkins/following{/other_user}","gists_url":"https://api.github.com/users/AmplabJenkins/gists{/gist_id}","starred_url":"https://api.github.com/users/AmplabJenkins/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AmplabJenkins/subscriptions","organizations_url":"https://api.github.com/users/AmplabJenkins/orgs","repos_url":"https://api.github.com/users/AmplabJenkins/repos","events_url":"https://api.github.com/users/AmplabJenkins/events{/privacy}","received_events_url":"https://api.github.com/users/AmplabJenkins/received_events","type":"User","site_admin":false},"created_at":"2015-08-04T21:52:57Z","updated_at":"2015-08-04T21:52:57Z","body":"Merged build started."}}