{"action":"created","issue":{"url":"https://api.github.com/repos/borgbackup/borg/issues/259","labels_url":"https://api.github.com/repos/borgbackup/borg/issues/259/labels{/name}","comments_url":"https://api.github.com/repos/borgbackup/borg/issues/259/comments","events_url":"https://api.github.com/repos/borgbackup/borg/issues/259/events","html_url":"https://github.com/borgbackup/borg/issues/259","id":110224589,"number":259,"title":"slow borg check (0.27 and prior releases)","user":{"login":"tgharold","id":4880437,"avatar_url":"https://avatars.githubusercontent.com/u/4880437?v=3","gravatar_id":"","url":"https://api.github.com/users/tgharold","html_url":"https://github.com/tgharold","followers_url":"https://api.github.com/users/tgharold/followers","following_url":"https://api.github.com/users/tgharold/following{/other_user}","gists_url":"https://api.github.com/users/tgharold/gists{/gist_id}","starred_url":"https://api.github.com/users/tgharold/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tgharold/subscriptions","organizations_url":"https://api.github.com/users/tgharold/orgs","repos_url":"https://api.github.com/users/tgharold/repos","events_url":"https://api.github.com/users/tgharold/events{/privacy}","received_events_url":"https://api.github.com/users/tgharold/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":12,"created_at":"2015-10-07T13:21:13Z","updated_at":"2015-10-31T20:07:15Z","closed_at":"2015-10-07T21:05:37Z","body":"On this repo (230GB, 87 archives), \"borg check\" consumes 100% of a CPU core, but doesn't have much disk traffic as measured by 'atop'.  And 'lsof /path/to/filesystem' shows it opening one file every few minutes.\r\n\r\nCtrl-C while running almost always causes it to emit this trace.\r\n\r\n```\r\nTraceback (most recent call last):\r\n  File \"/usr/lib64/python3.4/site-packages/borg/archiver.py\", line 1030, in main\r\n    exit_code = archiver.run(sys.argv[1:])\r\n  File \"/usr/lib64/python3.4/site-packages/borg/archiver.py\", line 986, in run\r\n    return args.func(args)\r\n  File \"/usr/lib64/python3.4/site-packages/borg/archiver.py\", line 95, in do_check\r\n    repository, repair=args.repair, archive=args.repository.archive, last=args.last):\r\n  File \"/usr/lib64/python3.4/site-packages/borg/archive.py\", line 627, in check\r\n    self.rebuild_refcounts(archive=archive, last=last)\r\n  File \"/usr/lib64/python3.4/site-packages/borg/archive.py\", line 795, in rebuild_refcounts\r\n    for item in robust_iterator(archive):\r\n  File \"/usr/lib64/python3.4/site-packages/borg/archive.py\", line 760, in robust_iterator\r\n    for item in unpacker:\r\n  File \"/usr/lib64/python3.4/site-packages/borg/archive.py\", line 607, in __next__\r\n    return next(self._unpacker)\r\n  File \"/usr/lib64/python3.4/site-packages/msgpack/fallback.py\", line 557, in next\r\n    ret = self._fb_unpack(EX_CONSTRUCT, None)\r\n  File \"/usr/lib64/python3.4/site-packages/msgpack/fallback.py\", line 538, in _fb_unpack\r\n    ret[key] = self._fb_unpack(EX_CONSTRUCT, write_bytes)\r\n  File \"/usr/lib64/python3.4/site-packages/msgpack/fallback.py\", line 517, in _fb_unpack\r\n    ret.append(self._fb_unpack(EX_CONSTRUCT, write_bytes))\r\n  File \"/usr/lib64/python3.4/site-packages/msgpack/fallback.py\", line 517, in _fb_unpack\r\n    ret.append(self._fb_unpack(EX_CONSTRUCT, write_bytes))\r\n  File \"/usr/lib64/python3.4/site-packages/msgpack/fallback.py\", line 498, in _fb_unpack\r\n    typ, n, obj = self._read_header(execute, write_bytes)\r\n  File \"/usr/lib64/python3.4/site-packages/msgpack/fallback.py\", line 346, in _read_header\r\n    c = self._fb_read(1, write_bytes)\r\n  File \"/usr/lib64/python3.4/site-packages/msgpack/fallback.py\", line 312, in _fb_read\r\n    return buffs[self._fb_buf_i][self._fb_buf_o - n:self._fb_buf_o]"},"comment":{"url":"https://api.github.com/repos/borgbackup/borg/issues/comments/152767409","html_url":"https://github.com/borgbackup/borg/issues/259#issuecomment-152767409","issue_url":"https://api.github.com/repos/borgbackup/borg/issues/259","id":152767409,"user":{"login":"ThomasWaldmann","id":356103,"avatar_url":"https://avatars.githubusercontent.com/u/356103?v=3","gravatar_id":"","url":"https://api.github.com/users/ThomasWaldmann","html_url":"https://github.com/ThomasWaldmann","followers_url":"https://api.github.com/users/ThomasWaldmann/followers","following_url":"https://api.github.com/users/ThomasWaldmann/following{/other_user}","gists_url":"https://api.github.com/users/ThomasWaldmann/gists{/gist_id}","starred_url":"https://api.github.com/users/ThomasWaldmann/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ThomasWaldmann/subscriptions","organizations_url":"https://api.github.com/users/ThomasWaldmann/orgs","repos_url":"https://api.github.com/users/ThomasWaldmann/repos","events_url":"https://api.github.com/users/ThomasWaldmann/events{/privacy}","received_events_url":"https://api.github.com/users/ThomasWaldmann/received_events","type":"User","site_admin":false},"created_at":"2015-10-31T20:07:15Z","updated_at":"2015-10-31T20:07:15Z","body":"btw, see #342 - I found what was missing on centos that led to msgpack not compiling the C (C++?) extension."}}