{"action":"created","issue":{"url":"https://api.github.com/repos/markfasheh/duperemove/issues/64","labels_url":"https://api.github.com/repos/markfasheh/duperemove/issues/64/labels{/name}","comments_url":"https://api.github.com/repos/markfasheh/duperemove/issues/64/comments","events_url":"https://api.github.com/repos/markfasheh/duperemove/issues/64/events","html_url":"https://github.com/markfasheh/duperemove/issues/64","id":78620799,"number":64,"title":"Crush on 4k blocks","user":{"login":"Nefelim4ag","id":3943460,"avatar_url":"https://avatars.githubusercontent.com/u/3943460?v=3","gravatar_id":"","url":"https://api.github.com/users/Nefelim4ag","html_url":"https://github.com/Nefelim4ag","followers_url":"https://api.github.com/users/Nefelim4ag/followers","following_url":"https://api.github.com/users/Nefelim4ag/following{/other_user}","gists_url":"https://api.github.com/users/Nefelim4ag/gists{/gist_id}","starred_url":"https://api.github.com/users/Nefelim4ag/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Nefelim4ag/subscriptions","organizations_url":"https://api.github.com/users/Nefelim4ag/orgs","repos_url":"https://api.github.com/users/Nefelim4ag/repos","events_url":"https://api.github.com/users/Nefelim4ag/events{/privacy}","received_events_url":"https://api.github.com/users/Nefelim4ag/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-05-20T17:38:38Z","updated_at":"2015-05-22T21:53:09Z","closed_at":null,"body":"FAILURE: Dedupe ioctl returns 9: Bad file descriptor\r\nERROR: dedupe.c:306\r\n/home/nefelim4ag/.local/share/Steam/SteamApps/common/Left 4 Dead 2/left4dead2_dlc2/maps/c8m3_sewers.bsp start block: 5114 (20946944)\r\n/home/nefelim4ag/.local/share/Steam/SteamApps/common/Left 4 Dead 2/left4dead2_dlc3/maps/c13m4_cutthroatcreek.bsp        start block: 2827 (11579392)\r\n[0x16aa800] Dedupe 1 extents with target: (20.0M, 4.0K), \"/home/nefelim4ag/.local/share/Steam/SteamApps/common/Left 4 Dead 2/left4dead2_dlc2/maps/c8m3_sewers.bsp\"\r\nadd ioctl request /home/nefelim4ag/.local/share/Steam/SteamApps/common/Left 4 Dead 2/left4dead2_dlc3/maps/c13m4_cutthroatcreek.bsp, off: 11579392, dest: 1\r\n/home/nefelim4ag/.local/share/Steam/SteamApps/common/Left 4 Dead 2/left4dead2_dlc3/maps/c10m3_ranchhouse.bsp    start block: 6742 (27615232)\r\n/home/nefelim4ag/.local/share/Steam/SteamApps/common/Left 4 Dead 2/left4dead2_dlc3/maps/c13m4_cutthroatcreek.bsp        start block: 3950 (16179200)\r\n[0x16aa800] Dedupe 1 extents with target: (26.3M, 4.0K), \"/home/nefelim4ag/.local/share/Steam/SteamApps/common/Left 4 Dead 2/left4dead2_dlc3/maps/c10m3_ranchhouse.bsp\"\r\nadd ioctl request /home/nefelim4ag/.local/share/Steam/SteamApps/common/Left 4 Dead 2/left4dead2_dlc3/maps/c13m4_cutthroatcreek.bsp, off: 16179200, dest: 1\r\n[stack trace follows]\r\nduperemove(print_stack_trace+0x2b) [0x407f3b]\r\nduperemove(pop_one_dedupe_result+0xf0) [0x406aa0]\r\nduperemove() [0x409ea3]\r\n/lib/x86_64-linux-gnu/libglib-2.0.so.0(+0x712e8) [0x7f56c1b7c2e8]\r\n/lib/x86_64-linux-gnu/libglib-2.0.so.0(+0x70955) [0x7f56c1b7b955]\r\n/lib/x86_64-linux-gnu/libpthread.so.0(+0x76aa) [0x7f56c0cee6aa]\r\n/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f56c1278eed]\r\n\r\nI've try to dedup data, with 128k block, all works fine, but duperemove crushed for 32k-4k blocks.\r\nSay me, if i can help debug this. i've use master branch.\r\n\r\nPS:\r\nUbuntu 15.04 x64"},"comment":{"url":"https://api.github.com/repos/markfasheh/duperemove/issues/comments/104784461","html_url":"https://github.com/markfasheh/duperemove/issues/64#issuecomment-104784461","issue_url":"https://api.github.com/repos/markfasheh/duperemove/issues/64","id":104784461,"user":{"login":"markfasheh","id":2341925,"avatar_url":"https://avatars.githubusercontent.com/u/2341925?v=3","gravatar_id":"","url":"https://api.github.com/users/markfasheh","html_url":"https://github.com/markfasheh","followers_url":"https://api.github.com/users/markfasheh/followers","following_url":"https://api.github.com/users/markfasheh/following{/other_user}","gists_url":"https://api.github.com/users/markfasheh/gists{/gist_id}","starred_url":"https://api.github.com/users/markfasheh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/markfasheh/subscriptions","organizations_url":"https://api.github.com/users/markfasheh/orgs","repos_url":"https://api.github.com/users/markfasheh/repos","events_url":"https://api.github.com/users/markfasheh/events{/privacy}","received_events_url":"https://api.github.com/users/markfasheh/received_events","type":"User","site_admin":false},"created_at":"2015-05-22T21:53:09Z","updated_at":"2015-05-22T21:53:09Z","body":"That looks like this line in pop_one_dedupe_result():\r\n\r\nhttps://github.com/markfasheh/duperemove/blob/master/dedupe.c#L306\r\n\r\nSo there's two things here, one is that we're aborting when we shouldn't and the 2nd is that you are getting EBADF from the dedupe ioctl. Presumably you aren't getting EBADF on the same files for 128k blocks.\r\n\r\nCan you grab the 'issue#64' branch of duperemove.git and try that out?\r\n\r\nhttps://github.com/markfasheh/duperemove/tree/issue%2364\r\n\r\nIt has two commits, one should prevent the program from aborting when it gets a dedupe failure. The other is printing some info on the dedupe ioctl every time it is run - I'd like to see what your failure looks like with the debugging on.\r\n\r\nSo basically we want to figure out why we're getting a bad file descriptor failure there, if you maybe put some debug prints inside dedupe_extent_list() (run_dedupe.c) to figure this out that would help."}}