{"action":"created","issue":{"url":"https://api.github.com/repos/pydata/pandas/issues/11600","labels_url":"https://api.github.com/repos/pydata/pandas/issues/11600/labels{/name}","comments_url":"https://api.github.com/repos/pydata/pandas/issues/11600/comments","events_url":"https://api.github.com/repos/pydata/pandas/issues/11600/events","html_url":"https://github.com/pydata/pandas/issues/11600","id":116880403,"number":11600,"title":"BUG - sparse dataframes lose multi-index column names","user":{"login":"Ezekiel-Kruglick","id":2335790,"avatar_url":"https://avatars.githubusercontent.com/u/2335790?v=3","gravatar_id":"","url":"https://api.github.com/users/Ezekiel-Kruglick","html_url":"https://github.com/Ezekiel-Kruglick","followers_url":"https://api.github.com/users/Ezekiel-Kruglick/followers","following_url":"https://api.github.com/users/Ezekiel-Kruglick/following{/other_user}","gists_url":"https://api.github.com/users/Ezekiel-Kruglick/gists{/gist_id}","starred_url":"https://api.github.com/users/Ezekiel-Kruglick/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Ezekiel-Kruglick/subscriptions","organizations_url":"https://api.github.com/users/Ezekiel-Kruglick/orgs","repos_url":"https://api.github.com/users/Ezekiel-Kruglick/repos","events_url":"https://api.github.com/users/Ezekiel-Kruglick/events{/privacy}","received_events_url":"https://api.github.com/users/Ezekiel-Kruglick/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/pydata/pandas/labels/Bug","name":"Bug","color":"e10c02"},{"url":"https://api.github.com/repos/pydata/pandas/labels/Difficulty%20Intermediate","name":"Difficulty Intermediate","color":"fbca04"},{"url":"https://api.github.com/repos/pydata/pandas/labels/Effort%20Medium","name":"Effort Medium","color":"006b75"},{"url":"https://api.github.com/repos/pydata/pandas/labels/MultiIndex","name":"MultiIndex","color":"207de5"},{"url":"https://api.github.com/repos/pydata/pandas/labels/Sparse","name":"Sparse","color":"009800"}],"state":"open","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/pydata/pandas/milestones/32","html_url":"https://github.com/pydata/pandas/milestones/Next%20Major%20Release","labels_url":"https://api.github.com/repos/pydata/pandas/milestones/32/labels","id":933188,"number":32,"title":"Next Major Release","description":"after 0.16.1 of course","creator":{"login":"jreback","id":953992,"avatar_url":"https://avatars.githubusercontent.com/u/953992?v=3","gravatar_id":"","url":"https://api.github.com/users/jreback","html_url":"https://github.com/jreback","followers_url":"https://api.github.com/users/jreback/followers","following_url":"https://api.github.com/users/jreback/following{/other_user}","gists_url":"https://api.github.com/users/jreback/gists{/gist_id}","starred_url":"https://api.github.com/users/jreback/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jreback/subscriptions","organizations_url":"https://api.github.com/users/jreback/orgs","repos_url":"https://api.github.com/users/jreback/repos","events_url":"https://api.github.com/users/jreback/events{/privacy}","received_events_url":"https://api.github.com/users/jreback/received_events","type":"User","site_admin":false},"open_issues":943,"closed_issues":96,"state":"open","created_at":"2015-01-13T10:53:19Z","updated_at":"2015-11-14T00:31:37Z","due_on":"2016-01-31T05:00:00Z","closed_at":null},"comments":3,"created_at":"2015-11-14T00:21:57Z","updated_at":"2015-11-14T00:45:04Z","closed_at":null,"body":"From SO: http://stackoverflow.com/questions/33702198/do-python-pandas-sparse-dataframes-lose-multi-index-column-names-or-am-i-doing-i\r\n\r\nBug is simple in concept, multi-index with column level names loses those names when going into sparse dataframes.\r\n\r\nMinimal example - first create a multi-index dataframe:\r\n```\r\nIn[2]: import pandas as pd\r\nIn[3]: miindex = pd.MultiIndex.from_product([[\"x\",\"y\"], [\"10\",\"20\"]],names=['row-foo', 'row-bar'])\r\nmicol = pd.MultiIndex.from_product([['a','b','c'], [\"1\",\"2\"]],names=['col-foo', 'col-bar'])\r\ndf = pd.DataFrame(index=miindex, columns=micol).sortlevel().sortlevel(axis=1)\r\ndf = df.fillna(value=3.14)\r\ndf\r\nOut[3]: \r\ncol-foo             a           b           c      \r\ncol-bar             1     2     1     2     1     2\r\nrow-foo row-bar                                    \r\nx       10       3.14  3.14  3.14  3.14  3.14  3.14\r\n        20       3.14  3.14  3.14  3.14  3.14  3.14\r\ny       10       3.14  3.14  3.14  3.14  3.14  3.14\r\n        20       3.14  3.14  3.14  3.14  3.14  3.14\r\n```\r\nThis gives us a nice test multi-index with column and row level names. Now if I make a sparse matrix out of that and show it, the column level names are gone.\r\n```\r\nIn[4]: ds = df.to_sparse()\r\nds\r\nOut[4]: \r\n                    a           b           c      \r\n                    1     2     1     2     1     2\r\nrow-foo row-bar                                    \r\nx       10       3.14  3.14  3.14  3.14  3.14  3.14\r\n        20       3.14  3.14  3.14  3.14  3.14  3.14\r\ny       10       3.14  3.14  3.14  3.14  3.14  3.14\r\n        20       3.14  3.14  3.14  3.14  3.14  3.14\r\n```\r\nAnd if I convert the sparse version back to dense those level names are still gone.\r\n```\r\nIn[6]: ds.to_dense()\r\nOut[6]: \r\n                    a           b           c      \r\n                    1     2     1     2     1     2\r\nrow-foo row-bar                                    \r\nx       10       3.14  3.14  3.14  3.14  3.14  3.14\r\n        20       3.14  3.14  3.14  3.14  3.14  3.14\r\ny       10       3.14  3.14  3.14  3.14  3.14  3.14\r\n        20       3.14  3.14  3.14  3.14  3.14  3.14\r\n```\r\nI AM aware that displaying the sparse version calls to_dense() but the loss appears to be happening at the conversion to sparse. I'm exploring moving to sparse to reduce memory usage for a code base and my attempts to access the levels within the sparse dataframe generate \"KeyError: 'Level not found'\""},"comment":{"url":"https://api.github.com/repos/pydata/pandas/issues/comments/156600496","html_url":"https://github.com/pydata/pandas/issues/11600#issuecomment-156600496","issue_url":"https://api.github.com/repos/pydata/pandas/issues/11600","id":156600496,"user":{"login":"jreback","id":953992,"avatar_url":"https://avatars.githubusercontent.com/u/953992?v=3","gravatar_id":"","url":"https://api.github.com/users/jreback","html_url":"https://github.com/jreback","followers_url":"https://api.github.com/users/jreback/followers","following_url":"https://api.github.com/users/jreback/following{/other_user}","gists_url":"https://api.github.com/users/jreback/gists{/gist_id}","starred_url":"https://api.github.com/users/jreback/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jreback/subscriptions","organizations_url":"https://api.github.com/users/jreback/orgs","repos_url":"https://api.github.com/users/jreback/repos","events_url":"https://api.github.com/users/jreback/events{/privacy}","received_events_url":"https://api.github.com/users/jreback/received_events","type":"User","site_admin":false},"created_at":"2015-11-14T00:45:04Z","updated_at":"2015-11-14T00:45:04Z","body":"that would be great!\r\n\r\nthere are a bunch of sparse issues which are on the easier side as well (though I don't think this is terribly involved)\r\n\r\nlmk when u need help"}}