{"action":"opened","issue":{"url":"https://api.github.com/repos/lorien/grab/issues/151","labels_url":"https://api.github.com/repos/lorien/grab/issues/151/labels{/name}","comments_url":"https://api.github.com/repos/lorien/grab/issues/151/comments","events_url":"https://api.github.com/repos/lorien/grab/issues/151/events","html_url":"https://github.com/lorien/grab/issues/151","id":107426750,"number":151,"title":"Remove pickle / Marshall","user":{"login":"kevinlondon","id":1216869,"avatar_url":"https://avatars.githubusercontent.com/u/1216869?v=3","gravatar_id":"","url":"https://api.github.com/users/kevinlondon","html_url":"https://github.com/kevinlondon","followers_url":"https://api.github.com/users/kevinlondon/followers","following_url":"https://api.github.com/users/kevinlondon/following{/other_user}","gists_url":"https://api.github.com/users/kevinlondon/gists{/gist_id}","starred_url":"https://api.github.com/users/kevinlondon/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kevinlondon/subscriptions","organizations_url":"https://api.github.com/users/kevinlondon/orgs","repos_url":"https://api.github.com/users/kevinlondon/repos","events_url":"https://api.github.com/users/kevinlondon/events{/privacy}","received_events_url":"https://api.github.com/users/kevinlondon/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2015-09-21T00:23:46Z","updated_at":"2015-09-21T00:23:46Z","closed_at":null,"body":"As per [another ticket](https://github.com/lorien/grab/pull/150), I'm looking for potential security issues in popular Python projects with Bandit. It looks like this project uses `pickle` and `marshall` for loading / unloading data. I'd recommend considering other ways to transfer data between processes.\r\n\r\nHere's a few examples from the run:\r\n\r\n```\r\n>> Issue: Deserialization with the marshal module is possibly dangerous.\r\n   Severity: Medium   Confidence: High\r\n   Location: ./grab/spider/cache_backend/mysql.py:102\r\n101\t            dump = zlib.decompress(val)\r\n102\t            return marshal.loads(dump)\r\n103\r\n\r\n>> Issue: Deserialization with the marshal module is possibly dangerous.\r\n   Severity: Medium   Confidence: High\r\n   Location: ./grab/spider/cache_backend/postgresql.py:92\r\n91\t            dump = zlib.decompress(val)\r\n92\t            return marshal.loads(dump)\r\n93\r\n\r\n>> Issue: Pickle library appears to be in use, possible security issue.\r\n   Severity: Medium   Confidence: High\r\n   Location: ./grab/spider/queue_backend/mongo.py:61\r\n60\t        else:\r\n61\t            return pickle.loads(item['task'])\r\n62\r\n```\r\nPickle is dangerous for loading and unloading files, as is marshall. Here's a [blog post by Lincoln Loop](https://lincolnloop.com/blog/2013/mar/22/playing-pickle-security/) about how it could be exploited, and a [blog post that I did](http://kevinlondon.com/2015/08/15/dangerous-python-functions-pt2.html) on the same thing.\r\n\r\nPerhaps JSON would work instead? Or XML, if you prefer?"}}