{"push_id":901705994,"size":1,"distinct_size":1,"ref":"refs/heads/master","head":"834e71489bf560302f9d743dff669df1134e9b74","before":"2aecda284e22ec608992b6221e2f5ffbd51fcd24","commits":[{"sha":"834e71489bf560302f9d743dff669df1134e9b74","author":{"email":"c437f8b7af73b0a10ecac061712ff9de6d41089e@databricks.com","name":"Davies Liu"},"message":"[SPARK-12213][SQL] use multiple partitions for single distinct query\n\nCurrently, we could generate different plans for query with single distinct (depends on spark.sql.specializeSingleDistinctAggPlanning), one works better on low cardinality columns, the other\nworks better for high cardinality column (default one).\n\nThis PR change to generate a single plan (three aggregations and two exchanges), which work better in both cases, then we could safely remove the flag `spark.sql.specializeSingleDistinctAggPlanning` (introduced in 1.6).\n\nFor a query like `SELECT COUNT(DISTINCT a) FROM table` will be\n```\nAGG-4 (count distinct)\n  Shuffle to a single reducer\n    Partial-AGG-3 (count distinct, no grouping)\n      Partial-AGG-2 (grouping on a)\n        Shuffle by a\n          Partial-AGG-1 (grouping on a)\n```\n\nThis PR also includes large refactor for aggregation (reduce 500+ lines of code)\n\ncc yhuai nongli marmbrus\n\nAuthor: Davies Liu <davies@databricks.com>\n\nCloses #10228 from davies/single_distinct.","distinct":true,"url":"https://api.github.com/repos/apache/spark/commits/834e71489bf560302f9d743dff669df1134e9b74"}]}