{"push_id":750167170,"size":1,"distinct_size":1,"ref":"refs/heads/branch-1.5","head":"78f168e97238316e33ce0d3763ba655603928c32","before":"e24b976506dd8563e4fe9cc295c756a1ce979e0d","commits":[{"sha":"78f168e97238316e33ce0d3763ba655603928c32","author":{"email":"affa8bc277be808c7f93de9249c2a75ded11876a@gmail.com","name":"Burak Yavuz"},"message":"[SPARK-9615] [SPARK-9616] [SQL] [MLLIB] Bugs related to FrequentItems when merging and with Tungsten\n\nIn short:\n1- FrequentItems should not use the InternalRow representation, because the keys in the map get messed up. For example, every key in the Map correspond to the very last element observed in the partition, when the elements are strings.\n\n2- Merging two partitions had a bug:\n\n**Existing behavior with size 3**\nPartition A -> Map(1 -> 3, 2 -> 3, 3 -> 4)\nPartition B -> Map(4 -> 25)\nResult -> Map()\n\n**Correct Behavior:**\nPartition A -> Map(1 -> 3, 2 -> 3, 3 -> 4)\nPartition B -> Map(4 -> 25)\nResult -> Map(3 -> 1, 4 -> 22)\n\ncc mengxr rxin JoshRosen\n\nAuthor: Burak Yavuz <brkyvz@gmail.com>\n\nCloses #7945 from brkyvz/freq-fix and squashes the following commits:\n\n07fa001 [Burak Yavuz] address 2\n1dc61a8 [Burak Yavuz] address 1\n506753e [Burak Yavuz] fixed and added reg test\n47bfd50 [Burak Yavuz] pushing\n\n(cherry picked from commit 98e69467d4fda2c26a951409b5b7c6f1e9345ce4)\nSigned-off-by: Xiangrui Meng <meng@databricks.com>","distinct":true,"url":"https://api.github.com/repos/apache/spark/commits/78f168e97238316e33ce0d3763ba655603928c32"}]}