{"action":"created","issue":{"url":"https://api.github.com/repos/zfsonlinux/zfs/issues/1548","labels_url":"https://api.github.com/repos/zfsonlinux/zfs/issues/1548/labels{/name}","comments_url":"https://api.github.com/repos/zfsonlinux/zfs/issues/1548/comments","events_url":"https://api.github.com/repos/zfsonlinux/zfs/issues/1548/events","html_url":"https://github.com/zfsonlinux/zfs/issues/1548","id":16005348,"number":1548,"title":"Deleting Files Doesn't Free Space, unless I unmount the filesystem","user":{"login":"fabiokorbes","id":4823491,"avatar_url":"https://avatars.githubusercontent.com/u/4823491?v=3","gravatar_id":"","url":"https://api.github.com/users/fabiokorbes","html_url":"https://github.com/fabiokorbes","followers_url":"https://api.github.com/users/fabiokorbes/followers","following_url":"https://api.github.com/users/fabiokorbes/following{/other_user}","gists_url":"https://api.github.com/users/fabiokorbes/gists{/gist_id}","starred_url":"https://api.github.com/users/fabiokorbes/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fabiokorbes/subscriptions","organizations_url":"https://api.github.com/users/fabiokorbes/orgs","repos_url":"https://api.github.com/users/fabiokorbes/repos","events_url":"https://api.github.com/users/fabiokorbes/events{/privacy}","received_events_url":"https://api.github.com/users/fabiokorbes/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/zfsonlinux/zfs/labels/Bug%20-%20Minor","name":"Bug - Minor","color":"e11d21"},{"url":"https://api.github.com/repos/zfsonlinux/zfs/labels/Difficulty%20-%20Medium","name":"Difficulty - Medium","color":"207de5"},{"url":"https://api.github.com/repos/zfsonlinux/zfs/labels/Memory%20Management","name":"Memory Management","color":"ffff33"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":61,"created_at":"2013-06-25T22:26:19Z","updated_at":"2015-07-14T15:09:02Z","closed_at":null,"body":"We are experiencing the same problem reported in issue https://github.com/zfsonlinux/zfs/issues/1188 . When we delete a file, its disk space is not free'd.\r\n\r\nWe are testing the use of a ZFS volume to store logs. We delete old logs in the same pace we write new ones. So, the disk usage should be constant in a short term. But it isn't. it always increase (as reported by df or zfs list commands). Although, the disk usage reported by du command remains constant.\r\n\r\nIt takes a full month to reach 100% of the disk. So I don't think it is an asynchronous-delete or recently-freed issues. I have already stopped the writing process for several hours, so it is not a load issue neither. And we aren't doing any snapshots.\r\n\r\nIt only releases the space of the deleted files when I unmount the filesystem. And it takes about 10 minutes to unmount.\r\n\r\nIs this a bug? or is there something I could do to make the deletions work?\r\n\r\nBesides, is there a command to manually release the space without I need to unmount the filesystem?\r\n\r\nHere, some info of my system:\r\n\r\n    # zdb \r\n    gfs:\r\n        version: 5000\r\n        name: 'gfs'\r\n        state: 0\r\n        txg: 4\r\n        pool_guid: 12548299384193180756\r\n        hostid: 3943305418\r\n        hostname: 'margari.tpn.terra.com'\r\n        vdev_children: 1\r\n        vdev_tree:\r\n            type: 'root'\r\n            id: 0\r\n            guid: 12548299384193180756\r\n            create_txg: 4\r\n            children[0]:\r\n                type: 'disk'\r\n                id: 0\r\n                guid: 776557494125630754\r\n                path: '/dev/sdb1'\r\n                whole_disk: 0\r\n                metaslab_array: 33\r\n                metaslab_shift: 32\r\n                ashift: 9\r\n                asize: 730802946048\r\n                is_log: 0\r\n                create_txg: 4\r\n        features_for_read:\r\n\r\n\r\n    # zpool status   \r\n      pool: gfs\r\n     state: ONLINE\r\n      scan: scrub repaired 0 in 4h6m with 0 errors on Sat Jun  8 00:42:59 2013\r\n    config:\r\n    \r\n            NAME        STATE     READ WRITE CKSUM\r\n            gfs         ONLINE       0     0     0\r\n              sdb1      ONLINE       0     0     0\r\n    \r\n    errors: No known data errors\r\n\r\n\r\n    # zfs list -t all\r\n    NAME   USED  AVAIL  REFER  MOUNTPOINT\r\n    gfs    364G   305G   364G  /gfs\r\n\r\n\r\n    # zfs get all\r\n    NAME  PROPERTY              VALUE                  SOURCE\r\n    gfs   type                  filesystem             -\r\n    gfs   creation              Tue Apr  9 13:13 2013  -\r\n    gfs   used                  364G                   -\r\n    gfs   available             305G                   -\r\n    gfs   referenced            364G                   -\r\n    gfs   compressratio         5.30x                  -\r\n    gfs   mounted               yes                    -\r\n    gfs   quota                 none                   default\r\n    gfs   reservation           none                   default\r\n    gfs   recordsize            64K                    local\r\n    gfs   mountpoint            /gfs                   default\r\n    gfs   sharenfs              off                    default\r\n    gfs   checksum              off                    local\r\n    gfs   compression           gzip-9                 local\r\n    gfs   atime                 off                    local\r\n    gfs   devices               on                     default\r\n    gfs   exec                  on                     default\r\n    gfs   setuid                on                     default\r\n    gfs   readonly              off                    default\r\n    gfs   zoned                 off                    default\r\n    gfs   snapdir               hidden                 default\r\n    gfs   aclinherit            restricted             default\r\n    gfs   canmount              on                     default\r\n    gfs   xattr                 on                     default\r\n    gfs   copies                1                      default\r\n    gfs   version               5                      -\r\n    gfs   utf8only              off                    -\r\n    gfs   normalization         none                   -\r\n    gfs   casesensitivity       sensitive              -\r\n    gfs   vscan                 off                    default\r\n    gfs   nbmand                off                    default\r\n    gfs   sharesmb              off                    default\r\n    gfs   refquota              none                   default\r\n    gfs   refreservation        none                   default\r\n    gfs   primarycache          all                    default\r\n    gfs   secondarycache        all                    default\r\n    gfs   usedbysnapshots       0                      -\r\n    gfs   usedbydataset         364G                   -\r\n    gfs   usedbychildren        88.2M                  -\r\n    gfs   usedbyrefreservation  0                      -\r\n    gfs   logbias               latency                default\r\n    gfs   dedup                 off                    default\r\n    gfs   mlslabel              none                   default\r\n    gfs   sync                  standard               default\r\n    gfs   refcompressratio      5.30x                  -\r\n    gfs   written               364G                   -\r\n    gfs   snapdev               hidden                 default\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/zfsonlinux/zfs/issues/comments/121276828","html_url":"https://github.com/zfsonlinux/zfs/issues/1548#issuecomment-121276828","issue_url":"https://api.github.com/repos/zfsonlinux/zfs/issues/1548","id":121276828,"user":{"login":"tobydarling","id":13333994,"avatar_url":"https://avatars.githubusercontent.com/u/13333994?v=3","gravatar_id":"","url":"https://api.github.com/users/tobydarling","html_url":"https://github.com/tobydarling","followers_url":"https://api.github.com/users/tobydarling/followers","following_url":"https://api.github.com/users/tobydarling/following{/other_user}","gists_url":"https://api.github.com/users/tobydarling/gists{/gist_id}","starred_url":"https://api.github.com/users/tobydarling/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tobydarling/subscriptions","organizations_url":"https://api.github.com/users/tobydarling/orgs","repos_url":"https://api.github.com/users/tobydarling/repos","events_url":"https://api.github.com/users/tobydarling/events{/privacy}","received_events_url":"https://api.github.com/users/tobydarling/received_events","type":"User","site_admin":false},"created_at":"2015-07-14T15:09:02Z","updated_at":"2015-07-14T15:09:02Z","body":"We're also experiencing this with 0.6.4.2 (upgraded from 0.6.4.1) and an NFSv3 exported volume using /etc/exports not sharenfs. This is on Scientific Linux 6.6 - 2.6.32-504.23.4.el6.x86_64. \r\n```\r\nSPL: Loaded module v0.6.4.1-1\r\nZFS: Loaded module v0.6.4.2-1, ZFS pool version 5000, ZFS filesystem version 5\r\nSPL: using hostid 0x00000000\r\n```\r\n(no changes to SPL between 0.6.4.1 and 0.6.4.2 so I'm guessing it didn't rebuild? https://github.com/zfsonlinux/spl/compare/spl-0.6.4-release...spl-0.6.4.2). \r\n\r\nI'm still trying to nail a reproducible test case. Currently, I still have 6765 ZAP entries in the delete queue  after deleting a directory structure of 76k files about 8 hours ago; directory was copied in yesterday and a snapshot taken during the night.\r\n"}}