{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/7841","labels_url":"https://api.github.com/repos/apache/spark/issues/7841/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/7841/comments","events_url":"https://api.github.com/repos/apache/spark/issues/7841/events","html_url":"https://github.com/apache/spark/pull/7841","id":98470018,"number":7841,"title":"[SPARK-8992] [SQL] Add pivot to dataframe api","user":{"login":"aray","id":981982,"avatar_url":"https://avatars.githubusercontent.com/u/981982?v=3","gravatar_id":"","url":"https://api.github.com/users/aray","html_url":"https://github.com/aray","followers_url":"https://api.github.com/users/aray/followers","following_url":"https://api.github.com/users/aray/following{/other_user}","gists_url":"https://api.github.com/users/aray/gists{/gist_id}","starred_url":"https://api.github.com/users/aray/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/aray/subscriptions","organizations_url":"https://api.github.com/users/aray/orgs","repos_url":"https://api.github.com/users/aray/repos","events_url":"https://api.github.com/users/aray/events{/privacy}","received_events_url":"https://api.github.com/users/aray/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":24,"created_at":"2015-07-31T20:57:42Z","updated_at":"2015-11-11T04:41:01Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/7841","html_url":"https://github.com/apache/spark/pull/7841","diff_url":"https://github.com/apache/spark/pull/7841.diff","patch_url":"https://github.com/apache/spark/pull/7841.patch"},"body":"This adds a pivot method to the dataframe api.\r\n\r\nFollowing the lead of cube and rollup this adds a Pivot operator that is translated into an Aggregate by the analyzer.\r\n\r\nCurrently the syntax is like:\r\n~~courseSales.pivot(Seq($\"year\"), $\"course\", Seq(\"dotNET\", \"Java\"), sum($\"earnings\"))~~\r\n\r\n~~Would we be interested in the following syntax also/alternatively? and~~\r\n\r\n    courseSales.groupBy($\"year\").pivot($\"course\", \"dotNET\", \"Java\").agg(sum($\"earnings\"))\r\n    //or\r\n    courseSales.groupBy($\"year\").pivot($\"course\").agg(sum($\"earnings\"))\r\n\r\nLater we can add it to `SQLParser`, but as Hive doesn't support it we cant add it there, right?\r\n\r\n~~Also what would be the suggested Java friendly method signature for this?~~"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/155661297","html_url":"https://github.com/apache/spark/pull/7841#issuecomment-155661297","issue_url":"https://api.github.com/repos/apache/spark/issues/7841","id":155661297,"user":{"login":"yhuai","id":2072857,"avatar_url":"https://avatars.githubusercontent.com/u/2072857?v=3","gravatar_id":"","url":"https://api.github.com/users/yhuai","html_url":"https://github.com/yhuai","followers_url":"https://api.github.com/users/yhuai/followers","following_url":"https://api.github.com/users/yhuai/following{/other_user}","gists_url":"https://api.github.com/users/yhuai/gists{/gist_id}","starred_url":"https://api.github.com/users/yhuai/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/yhuai/subscriptions","organizations_url":"https://api.github.com/users/yhuai/orgs","repos_url":"https://api.github.com/users/yhuai/repos","events_url":"https://api.github.com/users/yhuai/events{/privacy}","received_events_url":"https://api.github.com/users/yhuai/received_events","type":"User","site_admin":false},"created_at":"2015-11-11T04:41:01Z","updated_at":"2015-11-11T04:41:01Z","body":"@aray This is very cool! Here are a few things I'd like to discuss.\r\n1. Should we always ask users to provide pivot values?\r\n2. For the type of pivot values, should we just use String? Or, we should use Literal Column?\r\n3. For the column name of a pivot aggregation column, should we always show the aggregate function in a column name? (right now, we only show function name if we have more than 1 aggregations)"}}