{"action":"created","issue":{"url":"https://api.github.com/repos/pydata/pandas/issues/7879","labels_url":"https://api.github.com/repos/pydata/pandas/issues/7879/labels{/name}","comments_url":"https://api.github.com/repos/pydata/pandas/issues/7879/comments","events_url":"https://api.github.com/repos/pydata/pandas/issues/7879/events","html_url":"https://github.com/pydata/pandas/issues/7879","id":39093890,"number":7879,"title":"panel.to_frame() discards nan by default","user":{"login":"hensing","id":1055581,"avatar_url":"https://avatars.githubusercontent.com/u/1055581?v=3","gravatar_id":"","url":"https://api.github.com/users/hensing","html_url":"https://github.com/hensing","followers_url":"https://api.github.com/users/hensing/followers","following_url":"https://api.github.com/users/hensing/following{/other_user}","gists_url":"https://api.github.com/users/hensing/gists{/gist_id}","starred_url":"https://api.github.com/users/hensing/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hensing/subscriptions","organizations_url":"https://api.github.com/users/hensing/orgs","repos_url":"https://api.github.com/users/hensing/repos","events_url":"https://api.github.com/users/hensing/events{/privacy}","received_events_url":"https://api.github.com/users/hensing/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/pydata/pandas/labels/API%20Design","name":"API Design","color":"AD7FA8"},{"url":"https://api.github.com/repos/pydata/pandas/labels/Difficulty%20Novice","name":"Difficulty Novice","color":"fbca04"}],"state":"open","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/pydata/pandas/milestones/34","html_url":"https://github.com/pydata/pandas/milestones/0.17.0","labels_url":"https://api.github.com/repos/pydata/pandas/milestones/34/labels","id":1033710,"number":34,"title":"0.17.0","description":"","creator":{"login":"jreback","id":953992,"avatar_url":"https://avatars.githubusercontent.com/u/953992?v=3","gravatar_id":"","url":"https://api.github.com/users/jreback","html_url":"https://github.com/jreback","followers_url":"https://api.github.com/users/jreback/followers","following_url":"https://api.github.com/users/jreback/following{/other_user}","gists_url":"https://api.github.com/users/jreback/gists{/gist_id}","starred_url":"https://api.github.com/users/jreback/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jreback/subscriptions","organizations_url":"https://api.github.com/users/jreback/orgs","repos_url":"https://api.github.com/users/jreback/repos","events_url":"https://api.github.com/users/jreback/events{/privacy}","received_events_url":"https://api.github.com/users/jreback/received_events","type":"User","site_admin":false},"open_issues":136,"closed_issues":269,"state":"open","created_at":"2015-03-23T10:47:38Z","updated_at":"2015-08-16T13:50:01Z","due_on":"2015-08-31T04:00:00Z","closed_at":null},"comments":9,"created_at":"2014-07-30T13:54:09Z","updated_at":"2015-08-17T07:29:34Z","closed_at":null,"body":"As already described in issue #3690, panel.to_frame discards all minor entries with nan in the data, which can be very confusing. I believe there should be a warning, first time data is dropped or the opposite should be the default behavior.\r\n\r\nThe warning could be treated similar to a ZeroDivisionWarning in numpy only on the first occurrence.\r\n\r\nSee below for an example:\r\n\r\n```python\r\ndf1 = pd.DataFrame(np.random.randn(2, 3), columns=['A', 'B', 'C'],\r\n                   index=['foo', 'bar'])\r\ndf2 = pd.DataFrame(np.random.randn(2, 3), columns=['A', 'B', 'C'],\r\n                   index=['foo', 'bar'])\r\ndf2.loc['foo', 'B'] = np.nan\r\nmydict = {'df1': df1, 'df2': df2}\r\n\r\npd.Panel(mydict).to_frame()\r\n```\r\nOutput:  \r\n\r\nmajor | minor | df1 | df2\r\n------- | ------- | ---- | ----\r\nfoo | A | 1.9097545931480682 | -0.6710202447941566\r\nfoo | C | 1.3335254610685865 | 1.53372538551507\r\nbar | A | 0.3145550744497975 | -1.7221352144306152\r\nbar | B | -0.15681197178861878 | -1.2308510354641322\r\nbar | C | -0.09598971674309852 | -0.1268630728124487\r\n\r\n\r\nUsing filter_observations=False, nan won't be dropped:\r\n```python\r\npd.Panel(mydict).to_frame(filter_observations=False)\r\n```\r\nOutput:  \r\n\r\nmajor | minor | df1 | df2\r\n------- | ------- | ---- | ----\r\nfoo | A | 1.9097545931480682 | -0.6710202447941566\r\nfoo | B | 2.092552358833253 | \r\nfoo | C | 1.333525461068586 | 1.53372538551507\r\nbar | A | 0.3145550744497975 | -1.7221352144306152\r\nbar | B | -0.15681197178861878 | -1.2308510354641322\r\nbar | C | -0.09598971674309852 | -0.1268630728124487"},"comment":{"url":"https://api.github.com/repos/pydata/pandas/issues/comments/131705465","html_url":"https://github.com/pydata/pandas/issues/7879#issuecomment-131705465","issue_url":"https://api.github.com/repos/pydata/pandas/issues/7879","id":131705465,"user":{"login":"hensing","id":1055581,"avatar_url":"https://avatars.githubusercontent.com/u/1055581?v=3","gravatar_id":"","url":"https://api.github.com/users/hensing","html_url":"https://github.com/hensing","followers_url":"https://api.github.com/users/hensing/followers","following_url":"https://api.github.com/users/hensing/following{/other_user}","gists_url":"https://api.github.com/users/hensing/gists{/gist_id}","starred_url":"https://api.github.com/users/hensing/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hensing/subscriptions","organizations_url":"https://api.github.com/users/hensing/orgs","repos_url":"https://api.github.com/users/hensing/repos","events_url":"https://api.github.com/users/hensing/events{/privacy}","received_events_url":"https://api.github.com/users/hensing/received_events","type":"User","site_admin":false},"created_at":"2015-08-17T07:29:34Z","updated_at":"2015-08-17T07:29:34Z","body":"Is PR https://github.com/pydata/pandas/pull/8063 a solution or is it worth to write a new one or to change the default behavior?"}}