{"action":"created","issue":{"url":"https://api.github.com/repos/markfasheh/duperemove/issues/56","labels_url":"https://api.github.com/repos/markfasheh/duperemove/issues/56/labels{/name}","comments_url":"https://api.github.com/repos/markfasheh/duperemove/issues/56/comments","events_url":"https://api.github.com/repos/markfasheh/duperemove/issues/56/events","html_url":"https://github.com/markfasheh/duperemove/issues/56","id":67132337,"number":56,"title":"How fast should duperemove work?","user":{"login":"Rainmaker52","id":537352,"avatar_url":"https://avatars.githubusercontent.com/u/537352?v=3","gravatar_id":"","url":"https://api.github.com/users/Rainmaker52","html_url":"https://github.com/Rainmaker52","followers_url":"https://api.github.com/users/Rainmaker52/followers","following_url":"https://api.github.com/users/Rainmaker52/following{/other_user}","gists_url":"https://api.github.com/users/Rainmaker52/gists{/gist_id}","starred_url":"https://api.github.com/users/Rainmaker52/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Rainmaker52/subscriptions","organizations_url":"https://api.github.com/users/Rainmaker52/orgs","repos_url":"https://api.github.com/users/Rainmaker52/repos","events_url":"https://api.github.com/users/Rainmaker52/events{/privacy}","received_events_url":"https://api.github.com/users/Rainmaker52/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":25,"created_at":"2015-04-08T13:38:20Z","updated_at":"2015-06-17T16:04:07Z","closed_at":null,"body":"On my system, nothing seems to get dedupped when using duperemove (not even with ./show-shared-extents as shown in issue #55 )\r\n\r\nI have a volume filled with 100 GB of data. Duperemove calculates csum's, calculates shared extents, and then just hangs. It will still consume 100% (not 400%, but around 100%) of CPU.\r\n\r\nOutput I got is:\r\n> ./duperemove -rdhv -b 4k /sub/virtualbox\r\n> Warning: murmur3 support is experimental!\r\n> Using 4K blocks\r\n> Using hash: murmur3\r\n> Using 4 threads for file hashing phase\r\n> csum: /sub/virtualbox/ISO/Windows/en_windows_8.1_enterprise_with_update_x64_dvd_6054382.iso \t> [1/51] (1.96%)\r\n> ...\r\n> csum: /sub/virtualbox/ThinLic/ThinLic.vbox \t[51/51] (100.00%)\r\n> Hashing completed. Calculating duplicate extents - this may take some time.\r\n> [########################################]     \r\n> Search completed with no errors.       \r\n\r\nEven after waiting for several hours, nothing more seems to be happening.\r\n\r\nEven strace does not show any activity from the process (well, in relation to syscalls):\r\n> strace -s 1500 -f -p `pidof duperemove`\r\n> Process 2877 attached\r\n> ...\r\n> ...\r\n> ^CProcess 2877 detached\r\n\r\nAm I doing something wrong, or is there something broken?\r\nI'm using a git clone.\r\n\r\n(one more thing, I had to edit the Makefile to allow for compile, ld complained about \"undefined reference to `log'\"): \r\n> $ git diff\r\n> diff --git a/Makefile b/Makefile\r\n> index 8f03fb3..f98b99e 100644\r\n> --- a/Makefile\r\n> +++ b/Makefile\r\n> @@ -41,7 +41,7 @@ csum_test_obj = $(hash_obj) util.o csum.o\r\n> progs = duperemove hashstats btrfs-extent-same show-shared-extents csum-test\r\n> \r\n> glib_CFLAGS=$(shell pkg-config --cflags glib-2.0)\r\n> -glib_LIBS=$(shell pkg-config --libs glib-2.0)\r\n> +glib_LIBS=$(shell pkg-config --libs glib-2.0) -lm\r\n> \r\n> override CFLAGS += -D_FILE_OFFSET_BITS=64 -DVERSTRING=\\\"$(RELEASE)\\\" \\\r\n>        $(hash_CFLAGS) $(glib_CFLAGS) -rdynamic\r\n>"},"comment":{"url":"https://api.github.com/repos/markfasheh/duperemove/issues/comments/112861220","html_url":"https://github.com/markfasheh/duperemove/issues/56#issuecomment-112861220","issue_url":"https://api.github.com/repos/markfasheh/duperemove/issues/56","id":112861220,"user":{"login":"markfasheh","id":2341925,"avatar_url":"https://avatars.githubusercontent.com/u/2341925?v=3","gravatar_id":"","url":"https://api.github.com/users/markfasheh","html_url":"https://github.com/markfasheh","followers_url":"https://api.github.com/users/markfasheh/followers","following_url":"https://api.github.com/users/markfasheh/following{/other_user}","gists_url":"https://api.github.com/users/markfasheh/gists{/gist_id}","starred_url":"https://api.github.com/users/markfasheh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/markfasheh/subscriptions","organizations_url":"https://api.github.com/users/markfasheh/orgs","repos_url":"https://api.github.com/users/markfasheh/repos","events_url":"https://api.github.com/users/markfasheh/events{/privacy}","received_events_url":"https://api.github.com/users/markfasheh/received_events","type":"User","site_admin":false},"created_at":"2015-06-17T16:04:07Z","updated_at":"2015-06-17T16:04:07Z","body":"Juliantaylor, thanks for debugging this. I'll have a branch up shortly with your fix for everyone to test."}}