{"action":"created","issue":{"url":"https://api.github.com/repos/zfsonlinux/zfs/issues/3808","labels_url":"https://api.github.com/repos/zfsonlinux/zfs/issues/3808/labels{/name}","comments_url":"https://api.github.com/repos/zfsonlinux/zfs/issues/3808/comments","events_url":"https://api.github.com/repos/zfsonlinux/zfs/issues/3808/events","html_url":"https://github.com/zfsonlinux/zfs/issues/3808","id":107437943,"number":3808,"title":"Problems with large rsync's in 0.6.5","user":{"login":"rgmiller","id":1301479,"avatar_url":"https://avatars.githubusercontent.com/u/1301479?v=3","gravatar_id":"","url":"https://api.github.com/users/rgmiller","html_url":"https://github.com/rgmiller","followers_url":"https://api.github.com/users/rgmiller/followers","following_url":"https://api.github.com/users/rgmiller/following{/other_user}","gists_url":"https://api.github.com/users/rgmiller/gists{/gist_id}","starred_url":"https://api.github.com/users/rgmiller/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rgmiller/subscriptions","organizations_url":"https://api.github.com/users/rgmiller/orgs","repos_url":"https://api.github.com/users/rgmiller/repos","events_url":"https://api.github.com/users/rgmiller/events{/privacy}","received_events_url":"https://api.github.com/users/rgmiller/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":5,"created_at":"2015-09-21T03:27:21Z","updated_at":"2015-09-21T12:50:38Z","closed_at":null,"body":"I upgraded to 0.6.5 when it became available for CentOS 7 recently and I've begun to notice problems with large rsync transfers.  (I'm using zfs as the main storage for a BackupPC server.)  Small rsync's (ie: incremental backups) seem to be OK, but large ones (from the full backups) end up hanging and basically locking up the computer.\r\n\r\nI manged to get a stack trace from dmesg this evening, though, and I'll paste it below.  The traces all seem to be in a mutex locking function, so maybe this is some kind of deadlock scenario?\r\n\r\nI think I can trigger this situation fairly reliably by kicking off a full backup, so if there's something specific you want me to try, just let me know."},"comment":{"url":"https://api.github.com/repos/zfsonlinux/zfs/issues/comments/141965489","html_url":"https://github.com/zfsonlinux/zfs/issues/3808#issuecomment-141965489","issue_url":"https://api.github.com/repos/zfsonlinux/zfs/issues/3808","id":141965489,"user":{"login":"dweeezil","id":1432005,"avatar_url":"https://avatars.githubusercontent.com/u/1432005?v=3","gravatar_id":"","url":"https://api.github.com/users/dweeezil","html_url":"https://github.com/dweeezil","followers_url":"https://api.github.com/users/dweeezil/followers","following_url":"https://api.github.com/users/dweeezil/following{/other_user}","gists_url":"https://api.github.com/users/dweeezil/gists{/gist_id}","starred_url":"https://api.github.com/users/dweeezil/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dweeezil/subscriptions","organizations_url":"https://api.github.com/users/dweeezil/orgs","repos_url":"https://api.github.com/users/dweeezil/repos","events_url":"https://api.github.com/users/dweeezil/events{/privacy}","received_events_url":"https://api.github.com/users/dweeezil/received_events","type":"User","site_admin":false},"created_at":"2015-09-21T12:50:38Z","updated_at":"2015-09-21T12:50:38Z","body":"@rgmiller Does the target system have lots of files with xattrs?  If so, is ```xattr=sa``` set?  This deadlock seems like it would be a lot more likely when there are lots of dir-style xattrs.  The reason I ask it because a bug in 0.6.5 caused ```xattr=sa``` to not be enabled properly.  If this is the cause of your issue, there should be a fix coming very shortly in a point release which will cause ```xattr=sa``` to be properly enabled.\r\n\r\nThat said, however, this is clearly a deadlock situation which could occur in any case.  I'm not sure why, other than the above, it would be more likely in 0.6.5 than in earlier releases.  Or for that matter, why it might be more likely under kernels newer than 3.17.7.\r\n\r\nThese stack traces are very useful.  In summary, you've shown a deadlock which can occur during kswapd-invoked inode cache pruning due to the IO required during management of the unlinked set (f.k.a. \"delete queue\").   Actually, in this particular case, it's due to the waiting for dependent IO operations (parent zio)."}}