{"action":"created","issue":{"url":"https://api.github.com/repos/zfsonlinux/zfs/issues/3010","labels_url":"https://api.github.com/repos/zfsonlinux/zfs/issues/3010/labels{/name}","comments_url":"https://api.github.com/repos/zfsonlinux/zfs/issues/3010/comments","events_url":"https://api.github.com/repos/zfsonlinux/zfs/issues/3010/events","html_url":"https://github.com/zfsonlinux/zfs/issues/3010","id":54078073,"number":3010,"title":"Lost pools after many successful send/receives","user":{"login":"cousins","id":4643737,"avatar_url":"https://avatars.githubusercontent.com/u/4643737?v=3","gravatar_id":"","url":"https://api.github.com/users/cousins","html_url":"https://github.com/cousins","followers_url":"https://api.github.com/users/cousins/followers","following_url":"https://api.github.com/users/cousins/following{/other_user}","gists_url":"https://api.github.com/users/cousins/gists{/gist_id}","starred_url":"https://api.github.com/users/cousins/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cousins/subscriptions","organizations_url":"https://api.github.com/users/cousins/orgs","repos_url":"https://api.github.com/users/cousins/repos","events_url":"https://api.github.com/users/cousins/events{/privacy}","received_events_url":"https://api.github.com/users/cousins/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":12,"created_at":"2015-01-12T16:57:56Z","updated_at":"2015-01-15T15:04:11Z","closed_at":null,"body":"I recently (December) set up a new server to backup our primary ZFS server. It has CentOS 6.6 and ZFS/SPL version 0.6.3. I set up a big pool (similar to the primary pool) and then used send/receive to send snapshots over to the backup server. After the initial snapshots were done I sent incrementals over. All seemed fine. \r\n\r\nOn Friday (always on a Friday or holiday...) I noticed that the daily incremental sends weren't working anymore. Checking on the new server now I don't see  some of the pools and I find no reference to any of the snapshots on the backup server.  zpool status shows no problems and dmesg also shows nothing. I thought maybe there was just some inconsistency so I rebooted but things look exactly the same. I am hesitant to do anything until I get some advice.\r\n\r\nIn looking at the history I see that I didn't actually create a sub-pool on the backup server. So for instance I did:\r\n\r\n    zfs send pool2/os-grizzly@2014-12-18 | ssh nfs2-ib zfs receive -Fduv pool3\r\n\r\nAnd then subsequently:\r\n\r\n    zfs send -R -i pool2/os-grizzly@2014-12-18 pool2/os-grizzly@2014-12-19 | ssh nfs2-ib zfs receive -Fduv pool3\r\n\r\nAnd all seemed to work. I remember seeing /pool3/os-grizzly on the backup server and I could see snapshots listed on the backup server for pool3/os-grizzly. I have backup logs showing that it sent/received the streams successfully. \r\n\r\nI have other pools that are still showing up but they are pools that show a \"create\" entry in the zpool history log. Since it seemed to work without doing a create I figured I was all set. Now I'm wondering what happened. The data went somewhere and I wonder where it went. Are there any steps I can take to see what happened? Since this is a backup I'm not worried about the data. I'm more worried about pools vanishing when I really need them so I'd like to know what happened. \r\n\r\nI can supply more detailed information but for now I'm just looking for higher level information about what might have happened.\r\n \r\nThanks a lot for any advice.\r\n\r\nSteve"},"comment":{"url":"https://api.github.com/repos/zfsonlinux/zfs/issues/comments/70098284","html_url":"https://github.com/zfsonlinux/zfs/issues/3010#issuecomment-70098284","issue_url":"https://api.github.com/repos/zfsonlinux/zfs/issues/3010","id":70098284,"user":{"login":"cousins","id":4643737,"avatar_url":"https://avatars.githubusercontent.com/u/4643737?v=3","gravatar_id":"","url":"https://api.github.com/users/cousins","html_url":"https://github.com/cousins","followers_url":"https://api.github.com/users/cousins/followers","following_url":"https://api.github.com/users/cousins/following{/other_user}","gists_url":"https://api.github.com/users/cousins/gists{/gist_id}","starred_url":"https://api.github.com/users/cousins/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cousins/subscriptions","organizations_url":"https://api.github.com/users/cousins/orgs","repos_url":"https://api.github.com/users/cousins/repos","events_url":"https://api.github.com/users/cousins/events{/privacy}","received_events_url":"https://api.github.com/users/cousins/received_events","type":"User","site_admin":false},"created_at":"2015-01-15T15:04:11Z","updated_at":"2015-01-15T15:04:11Z","body":"@eborisch Thanks for the information. The thing is that the snapshots and pools still exist on the sending side so my understanding is that the receiving side shouldn't have deleted these if -F is working correctly. That said, I'm going to see about not using -F anymore\r\n\r\nThe current priority is to try to roll back to a txg that is before the pools were destroyed. Any recommendations on how best to do that?\r\n"}}