{"action":"created","issue":{"url":"https://api.github.com/repos/pydata/pandas/issues/5839","labels_url":"https://api.github.com/repos/pydata/pandas/issues/5839/labels{/name}","comments_url":"https://api.github.com/repos/pydata/pandas/issues/5839/comments","events_url":"https://api.github.com/repos/pydata/pandas/issues/5839/events","html_url":"https://github.com/pydata/pandas/issues/5839","id":25035664,"number":5839,"title":"df.groupby().apply() with only one group returns wrong shape!","user":{"login":"dragoljub","id":2701562,"avatar_url":"https://avatars.githubusercontent.com/u/2701562?v=3","gravatar_id":"","url":"https://api.github.com/users/dragoljub","html_url":"https://github.com/dragoljub","followers_url":"https://api.github.com/users/dragoljub/followers","following_url":"https://api.github.com/users/dragoljub/following{/other_user}","gists_url":"https://api.github.com/users/dragoljub/gists{/gist_id}","starred_url":"https://api.github.com/users/dragoljub/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dragoljub/subscriptions","organizations_url":"https://api.github.com/users/dragoljub/orgs","repos_url":"https://api.github.com/users/dragoljub/repos","events_url":"https://api.github.com/users/dragoljub/events{/privacy}","received_events_url":"https://api.github.com/users/dragoljub/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/pydata/pandas/labels/Bug","name":"Bug","color":"e10c02"},{"url":"https://api.github.com/repos/pydata/pandas/labels/Groupby","name":"Groupby","color":"729FCF"},{"url":"https://api.github.com/repos/pydata/pandas/labels/MultiIndex","name":"MultiIndex","color":"207de5"},{"url":"https://api.github.com/repos/pydata/pandas/labels/Won't%20Fix","name":"Won't Fix","color":"5319e7"}],"state":"open","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/pydata/pandas/milestones/32","html_url":"https://github.com/pydata/pandas/milestones/Next%20Major%20Release","labels_url":"https://api.github.com/repos/pydata/pandas/milestones/32/labels","id":933188,"number":32,"title":"Next Major Release","description":"after 0.16.1 of course","creator":{"login":"jreback","id":953992,"avatar_url":"https://avatars.githubusercontent.com/u/953992?v=3","gravatar_id":"","url":"https://api.github.com/users/jreback","html_url":"https://github.com/jreback","followers_url":"https://api.github.com/users/jreback/followers","following_url":"https://api.github.com/users/jreback/following{/other_user}","gists_url":"https://api.github.com/users/jreback/gists{/gist_id}","starred_url":"https://api.github.com/users/jreback/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jreback/subscriptions","organizations_url":"https://api.github.com/users/jreback/orgs","repos_url":"https://api.github.com/users/jreback/repos","events_url":"https://api.github.com/users/jreback/events{/privacy}","received_events_url":"https://api.github.com/users/jreback/received_events","type":"User","site_admin":false},"open_issues":910,"closed_issues":73,"state":"open","created_at":"2015-01-13T10:53:19Z","updated_at":"2015-10-02T14:16:01Z","due_on":"2016-01-31T05:00:00Z","closed_at":null},"comments":16,"created_at":"2014-01-03T21:02:49Z","updated_at":"2015-10-02T18:03:44Z","closed_at":null,"body":"I have reached a corner case in the wonderful groupby().apply() method. A groupby with only __one__ group causes the apply method to return the wrong output shape. Instead of a series with a multi-index the result is retunred as a DataFrame with last row index level as columns. :frowning: \r\n\r\n```python\r\n\r\nIn [1]: import numpy as np\r\n   ...: import pandas as pd\r\n   ...: from sklearn.cluster import DBSCAN as DBSCAN\r\n   ...: \r\n   ...: print pd.__version__\r\n   ...: \r\n   ...: # Generate Test DataFrame\r\n   ...: NUM_ROWS = 1000\r\n   ...: NUM_COLS = 10\r\n   ...: col_names = ['A'+num for num in map(str,np.arange(NUM_COLS).tolist())]\r\n   ...: index_cols = col_names[:5] \r\n   ...: \r\n   ...: # Set DataFrame to have 5 level Hierarchical Index.\r\n   ...: # Sort the index!\r\n   ...: df = pd.DataFrame(np.random.randint(5, size=(NUM_ROWS,NUM_COLS)), dtype=np.int64, columns=col_names)\r\n   ...: df = df.set_index(index_cols).sort_index()\r\n   ...: df\r\n   ...: \r\n   ...: # Group by first 4 index columns.\r\n   ...: grp = df.groupby(level=index_cols[:4])\r\n   ...: \r\n   ...: # Find index of largest group.\r\n   ...: big_loc = grp.size().idxmax()\r\n   ...: \r\n   ...: # Create function to apply clustering on groups\r\n   ...: def grp_func(df):\r\n   ...:     \"\"\"Run clustering on subgroup and return series of results.\"\"\"\r\n   ...:     db = DBSCAN(eps=1, min_samples=1, metric='euclidean').fit(df.values)\r\n   ...:     return pd.Series(db.labels_, name='cluster_id', index=df.index.get_level_values(4))\r\n   ...: \r\n0.12.0\r\n\r\nIn [2]: # Apply clustering on each subgroup of DataFrame\r\n   ...: out_good = grp.apply(grp_func)\r\n   ...: out_good\r\n   ...: out_good.shape\r\nOut[2]: (1000L,)\r\n\r\nIn [3]: # Select out biggest group wihile keeping index levels and try same apply\r\n   ...: out_bad = df[[big_loc == a[:4] for a in df.index.values]].groupby(level=index_cols[:4]).apply(grp_func)\r\n   ...: out_bad\r\n   ...: out_bad.shape\r\nOut[3]: (1, 7)\r\n\r\nIn [4]: out_good\r\nOut[4]: \r\nA0  A1  A2  A3  A4\r\n0   0   0   0   0     1\r\n                3     0\r\n            1   3     0\r\n            2   1     1\r\n                3     2\r\n                3     0\r\n            3   3     1\r\n                3     0\r\n        1   1   1     0\r\n            2   0     0\r\n                2     1\r\n                4     2\r\n            3   4     0\r\n                4     1\r\n            4   0     0\r\n...\r\n4   4   3   0   2     0\r\n            1   1     1\r\n                3     2\r\n                4     0\r\n            3   1     0\r\n                2     1\r\n            4   4     0\r\n        4   0   4     0\r\n            1   1     3\r\n                1     1\r\n                2     2\r\n                2     0\r\n            3   1     1\r\n                3     0\r\n            4   4     0\r\nName: cluster_id, Length: 1000, dtype: float64\r\n\r\nIn [5]: out_bad\r\n\r\nOut[5]: \r\nA4           1  1  2  3  3  3  4\r\nA0 A1 A2 A3                     \r\n3  0  0  3   6  5  3  0  1  4  2\r\n\r\n# If you stack the bad result it comes out looking OK, but now I need a workaround for this corner case to use apply.\r\nIn [17]: out_bad.stack()\r\n\r\nOut[17]: A0  A1  A2  A3  A4\r\n3   0   0   3   1     3\r\n                1     6\r\n                2     5\r\n                3     1\r\n                3     4\r\n                3     0\r\n                4     2\r\ndtype: float64\r\n```"},"comment":{"url":"https://api.github.com/repos/pydata/pandas/issues/comments/145106582","html_url":"https://github.com/pydata/pandas/issues/5839#issuecomment-145106582","issue_url":"https://api.github.com/repos/pydata/pandas/issues/5839","id":145106582,"user":{"login":"jerryatmda","id":1727968,"avatar_url":"https://avatars.githubusercontent.com/u/1727968?v=3","gravatar_id":"","url":"https://api.github.com/users/jerryatmda","html_url":"https://github.com/jerryatmda","followers_url":"https://api.github.com/users/jerryatmda/followers","following_url":"https://api.github.com/users/jerryatmda/following{/other_user}","gists_url":"https://api.github.com/users/jerryatmda/gists{/gist_id}","starred_url":"https://api.github.com/users/jerryatmda/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jerryatmda/subscriptions","organizations_url":"https://api.github.com/users/jerryatmda/orgs","repos_url":"https://api.github.com/users/jerryatmda/repos","events_url":"https://api.github.com/users/jerryatmda/events{/privacy}","received_events_url":"https://api.github.com/users/jerryatmda/received_events","type":"User","site_admin":false},"created_at":"2015-10-02T18:03:44Z","updated_at":"2015-10-02T18:03:44Z","body":"This issue is very similar to a behavior that I consider a bug or misfeature. Because\r\nI have a different use-case and a small package for reproducing it, I have submitted it\r\nas a separate issue: #11224"}}