{"action":"closed","issue":{"url":"https://api.github.com/repos/andresriancho/w3af/issues/8748","labels_url":"https://api.github.com/repos/andresriancho/w3af/issues/8748/labels{/name}","comments_url":"https://api.github.com/repos/andresriancho/w3af/issues/8748/comments","events_url":"https://api.github.com/repos/andresriancho/w3af/issues/8748/events","html_url":"https://github.com/andresriancho/w3af/issues/8748","id":60239141,"number":8748,"title":"MaybeEncodingError - PicklingError: Can't pickle dictproxy","user":{"login":"1d3df9903ad","id":3416690,"avatar_url":"https://avatars.githubusercontent.com/u/3416690?v=3","gravatar_id":"","url":"https://api.github.com/users/1d3df9903ad","html_url":"https://github.com/1d3df9903ad","followers_url":"https://api.github.com/users/1d3df9903ad/followers","following_url":"https://api.github.com/users/1d3df9903ad/following{/other_user}","gists_url":"https://api.github.com/users/1d3df9903ad/gists{/gist_id}","starred_url":"https://api.github.com/users/1d3df9903ad/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/1d3df9903ad/subscriptions","organizations_url":"https://api.github.com/users/1d3df9903ad/orgs","repos_url":"https://api.github.com/users/1d3df9903ad/repos","events_url":"https://api.github.com/users/1d3df9903ad/events{/privacy}","received_events_url":"https://api.github.com/users/1d3df9903ad/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/andresriancho/w3af/labels/bug","name":"bug","color":"e10c02"},{"url":"https://api.github.com/repos/andresriancho/w3af/labels/priority:low","name":"priority:low","color":"009800"}],"state":"closed","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/andresriancho/w3af/milestones/10","html_url":"https://github.com/andresriancho/w3af/milestones/1.6.50%20-%20Bug%20fixing%20after%201.6.0","labels_url":"https://api.github.com/repos/andresriancho/w3af/milestones/10/labels","id":614510,"number":10,"title":"1.6.50 - Bug fixing after 1.6.0","description":"Bug fixes after 1.6 release, this is a MUST-HAVE since people will start using 1.6 and report severity 1 bugs which need to be fixed and pushed to all users.","creator":{"login":"andresriancho","id":865200,"avatar_url":"https://avatars.githubusercontent.com/u/865200?v=3","gravatar_id":"","url":"https://api.github.com/users/andresriancho","html_url":"https://github.com/andresriancho","followers_url":"https://api.github.com/users/andresriancho/followers","following_url":"https://api.github.com/users/andresriancho/following{/other_user}","gists_url":"https://api.github.com/users/andresriancho/gists{/gist_id}","starred_url":"https://api.github.com/users/andresriancho/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/andresriancho/subscriptions","organizations_url":"https://api.github.com/users/andresriancho/orgs","repos_url":"https://api.github.com/users/andresriancho/repos","events_url":"https://api.github.com/users/andresriancho/events{/privacy}","received_events_url":"https://api.github.com/users/andresriancho/received_events","type":"User","site_admin":false},"open_issues":17,"closed_issues":366,"state":"open","created_at":"2014-03-30T17:44:39Z","updated_at":"2015-06-08T18:38:33Z","due_on":null,"closed_at":null},"comments":5,"created_at":"2015-03-08T03:33:07Z","updated_at":"2015-06-08T18:38:33Z","closed_at":"2015-06-08T18:38:33Z","body":"## History\r\n * In 6768bd8 I added debugging information that will help me identify which parser is breaking pickle.\r\n * 6768bd8 was merged into master\r\n * Waiting for new bug reports with detailed/improved MaybeEncodingError errors. \r\n * Got [an exception with details](https://github.com/andresriancho/w3af/issues/8963)!\r\n\r\n```\r\nError sending result: '<HTMLParser DocumentParser for \"<HTTPResponse\r\n | 200 | http://www.ensinosuperior.org.br/asesi.htm | id:383>\">'. Reason:\r\n 'PicklingError(\"Can't pickle <type 'dictproxy'>: attribute lookup __builtin__.\r\ndictproxy failed\",)\r\n```\r\n\r\nThis reproduces the exception:\r\n\r\n```\r\nfrom multiprocessing.queues import SimpleQueue\r\nsq = SimpleQueue()\r\nsq.put(type.__dict__)\r\n```\r\n\r\n * Still not enough information, now with https://github.com/andresriancho/w3af/commit/8ff0e0956203d90a2bceca98b6f8152216464e2e I'll get the attribute name in HTMLParser which is generating this issue\r\n\r\n * 23 Apr 2015: [No search results](https://github.com/andresriancho/w3af/search?q=%22Conflicting+attr%22&type=Issues&utf8=%E2%9C%93) for `Conflicting attr`\r\n\r\n * Lowering priority since we're not getting any reports for this\r\n\r\n * Exception with [more detail appears](https://github.com/andresriancho/w3af/issues/9648): A \"MaybeEncodingError\" exception was found while running infrastructure.finger_bing on \"Method: GET | http://domain/\". The exception was: \"Error sending result: '<w3af.core.data.parsers.parser_cache.ProcessDocumentParser object at 0x384ccd0>'. Reason: 'PicklingError(\"Can't pickle <class 'lxml.etree._ImmutableMapping'>: attribute lookup lxml.etree._ImmutableMapping failed\",)'\" at pool276.py:get():558.The full traceback is:\r\n  \r\n\r\n## Error message\r\n```\r\nError sending result: '<w3af.core.data.parsers.parser_cache.ProcessDocumentParser\r\nobject at 0xb8637ec>'. Reason: 'PicklingError(\"Can't pickle <type 'dictproxy'>:\r\nattribute lookup __builtin__.dictproxy failed\",)'\r\n```\r\n\r\n## Version Information\r\n```\r\n  Python version: 2.7.3 (default, Mar 14 2014, 11:57:14) [GCC 4.7.2]\r\n  GTK version: 2.24.10\r\n  PyGTK version: 2.24.0\r\n  w3af version:\r\n    w3af - Web Application Attack and Audit Framework\r\n    Version: 1.6.45\r\n    Revision: b7cffaa62a - 26 Фев 2015 13:30\r\n    Branch: master\r\n    Local changes: No\r\n    Author: Andres Riancho and the w3af team.\r\n```\r\n## Traceback\r\n```pytb\r\nA \"MaybeEncodingError\" exception was found while running crawl.web_spider on \"Method: GET | http://domain/themes/mobile/style/style.css\". The exception was: \"Error sending result: '<w3af.core.data.parsers.parser_cache.ProcessDocumentParser object at 0xb8637ec>'. Reason: 'PicklingError(\"Can't pickle <type 'dictproxy'>: attribute lookup __builtin__.dictproxy failed\",)'\" at pool276.py:get():558.The full traceback is:\r\n  File \"/root/w3af/w3af/core/controllers/core_helpers/consumers/crawl_infrastructure.py\", line 401, in _discover_worker\r\n    result = plugin.discover_wrapper(fuzzable_request)\r\n  File \"/root/w3af/w3af/core/controllers/plugins/crawl_plugin.py\", line 60, in crawl_wrapper\r\n    return self.crawl(fuzzable_request_copy)\r\n  File \"/root/w3af/w3af/plugins/crawl/web_spider.py\", line 112, in crawl\r\n    self._extract_html_forms(resp, fuzzable_req)\r\n  File \"/root/w3af/w3af/plugins/crawl/web_spider.py\", line 122, in _extract_html_forms\r\n    dp = parser_cache.dpc.get_document_parser_for(resp)\r\n  File \"/root/w3af/w3af/core/data/parsers/parser_cache.py\", line 246, in get_document_parser_for\r\n    hash_string)\r\n  File \"/root/w3af/w3af/core/data/parsers/parser_cache.py\", line 165, in _parse_http_response_in_worker\r\n    parser_output = result.get(timeout=self.PARSER_TIMEOUT)\r\n  File \"/root/w3af/w3af/core/controllers/threads/pool276.py\", line 558, in get\r\n    raise self._value\r\n\r\n```\r\n## Enabled Plugins\r\n```python\r\n{'attack': {},\r\n 'audit': {u'blind_sqli': <OptionList: eq_limit>,\r\n           u'sqli': <OptionList: >,\r\n           u'xss': <OptionList: persistent_xss>},\r\n 'auth': {},\r\n 'bruteforce': {},\r\n 'crawl': {u'web_spider': <OptionList: only_forward|follow_regex|ignore_regex>},\r\n 'evasion': {},\r\n 'grep': {},\r\n 'infrastructure': {},\r\n 'mangle': {},\r\n 'output': {u'console': <OptionList: verbose>}}\r\n\r\n```\r\n"}}