{"action":"created","issue":{"url":"https://api.github.com/repos/lpinner/metageta/issues/53","labels_url":"https://api.github.com/repos/lpinner/metageta/issues/53/labels{/name}","comments_url":"https://api.github.com/repos/lpinner/metageta/issues/53/comments","events_url":"https://api.github.com/repos/lpinner/metageta/issues/53/events","html_url":"https://github.com/lpinner/metageta/issues/53","id":118438083,"number":53,"title":"Error on compressed file","user":{"login":"alexlopespereira","id":6627726,"avatar_url":"https://avatars.githubusercontent.com/u/6627726?v=3","gravatar_id":"","url":"https://api.github.com/users/alexlopespereira","html_url":"https://github.com/alexlopespereira","followers_url":"https://api.github.com/users/alexlopespereira/followers","following_url":"https://api.github.com/users/alexlopespereira/following{/other_user}","gists_url":"https://api.github.com/users/alexlopespereira/gists{/gist_id}","starred_url":"https://api.github.com/users/alexlopespereira/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/alexlopespereira/subscriptions","organizations_url":"https://api.github.com/users/alexlopespereira/orgs","repos_url":"https://api.github.com/users/alexlopespereira/repos","events_url":"https://api.github.com/users/alexlopespereira/events{/privacy}","received_events_url":"https://api.github.com/users/alexlopespereira/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2015-11-23T18:04:56Z","updated_at":"2015-12-04T14:20:09Z","closed_at":null,"body":"Hi @lpinner,\r\nI have run the following command using metageta 1.4.0 and it returns an error. Using metageta 1.3.9 no such error appears while testing over the same images. I could delete the erroneous file, but I don't know which file is this. The filename was not logged into the log file.\r\nI would like to know whether you could verify and try to fix it (if it's necessary), or maybe handle the exception so that it does not stop the crawler.\r\n\r\nThank you very much,\r\nAlex\r\n\r\n> [alex@srvindice ~]$ runcrawler --debug -a -r -d /media/imagens/ -x indice\r\n> Unable to import Tix, tkFileDialog and/or tkMessageBox\r\n> 13:46:37\tDEBUG\t/usr/bin/runcrawler --debug -a -r -d /media/imagens/ -x indice\r\n> 13:46:37\tWARNING\tUse formatting objects such as font directly\r\n> Warning 6: Normalized/laundered field name: 'datecreated' to 'datecreate'\r\n> Warning 6: Normalized/laundered field name: 'datemodified' to 'datemodifi'\r\n> Warning 1: Field filepath of width 255 truncated to 254.\r\n> 13:46:37\tINFO\tSearching for files...\r\n> Traceback (most recent call last):\r\n>  File \"/usr/bin/runcrawler\", line 9, in <module>\r\n>    load_entry_point('MetaGETA==1.4.0', 'console_scripts', 'runcrawler')()\r\n>  File \"/usr/lib/python2.7/site-packages/MetaGETA-1.4.0-py2.7.egg/metageta/__runcrawler__.py\", line > 430, in main\r\n>     optvals.recurse,optvals.archive,optvals.excludes)\r\n>   File \"/usr/lib/python2.7/site-packages/MetaGETA-1.4.0-py2.7.egg/metageta/__runcrawler__.py\", line > 135, in execute\r\n>     Crawler=crawler.Crawler(dir,recurse=recurse,archive=archive,excludes=excludes)\r\n>   File \"/usr/lib/python2.7/site-packages/MetaGETA-1.4.0-py2.7.egg/metageta/crawler.py\", line 63, in  __init__\r\n>     recurse=recurse, archive=archive, excludes=excludes):\r\n>   File \"/usr/lib/python2.7/site-packages/MetaGETA-1.4.0-py2.7.egg/metageta/utilities.py\", line 552, in > rglob\r\n>     paths = archivelist(fullname)\r\n>   File \"/usr/lib/python2.7/site-packages/MetaGETA-1.4.0-py2.7.egg/metageta/utilities.py\", line 85, in archivelist\r\n>     lst=[ti.name for ti in tarfile.open(f,'r').getmembers() if ti.isfile()]\r\n>   File \"/usr/lib64/python2.7/tarfile.py\", line 1805, in getmembers\r\n>     self._load()        # all members, we first have to\r\n>   File \"/usr/lib64/python2.7/tarfile.py\", line 2380, in _load\r\n>     tarinfo = self.next()\r\n>   File \"/usr/lib64/python2.7/tarfile.py\", line 2315, in next\r\n>     self.fileobj.seek(self.offset)\r\n>   File \"/usr/lib64/python2.7/gzip.py\", line 434, in seek\r\n>     self.read(1024)\r\n>   File \"/usr/lib64/python2.7/gzip.py\", line 261, in read\r\n>     self._read(readsize)\r\n>   File \"/usr/lib64/python2.7/gzip.py\", line 308, in _read\r\n>     self._read_eof()\r\n>   File \"/usr/lib64/python2.7/gzip.py\", line 347, in _read_eof\r\n>     hex(self.crc)))\r\n> IOError: CRC check failed 0x1d823b02 != 0x1bcc2669L\r\n"},"comment":{"url":"https://api.github.com/repos/lpinner/metageta/issues/comments/161978657","html_url":"https://github.com/lpinner/metageta/issues/53#issuecomment-161978657","issue_url":"https://api.github.com/repos/lpinner/metageta/issues/53","id":161978657,"user":{"login":"alexlopespereira","id":6627726,"avatar_url":"https://avatars.githubusercontent.com/u/6627726?v=3","gravatar_id":"","url":"https://api.github.com/users/alexlopespereira","html_url":"https://github.com/alexlopespereira","followers_url":"https://api.github.com/users/alexlopespereira/followers","following_url":"https://api.github.com/users/alexlopespereira/following{/other_user}","gists_url":"https://api.github.com/users/alexlopespereira/gists{/gist_id}","starred_url":"https://api.github.com/users/alexlopespereira/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/alexlopespereira/subscriptions","organizations_url":"https://api.github.com/users/alexlopespereira/orgs","repos_url":"https://api.github.com/users/alexlopespereira/repos","events_url":"https://api.github.com/users/alexlopespereira/events{/privacy}","received_events_url":"https://api.github.com/users/alexlopespereira/received_events","type":"User","site_admin":false},"created_at":"2015-12-04T14:20:09Z","updated_at":"2015-12-04T14:20:09Z","body":"@lpinner I tested this new version and it works as expected. It handles the exception and continuous crawling.\r\n\r\nThank you very much.\r\nAlex"}}