{"action":"created","issue":{"url":"https://api.github.com/repos/binux/pyspider/issues/178","labels_url":"https://api.github.com/repos/binux/pyspider/issues/178/labels{/name}","comments_url":"https://api.github.com/repos/binux/pyspider/issues/178/comments","events_url":"https://api.github.com/repos/binux/pyspider/issues/178/events","html_url":"https://github.com/binux/pyspider/issues/178","id":67941257,"number":178,"title":"Trying to use MongoDB as database, getting exceptions","user":{"login":"virtuman","id":1087996,"avatar_url":"https://avatars.githubusercontent.com/u/1087996?v=3","gravatar_id":"","url":"https://api.github.com/users/virtuman","html_url":"https://github.com/virtuman","followers_url":"https://api.github.com/users/virtuman/followers","following_url":"https://api.github.com/users/virtuman/following{/other_user}","gists_url":"https://api.github.com/users/virtuman/gists{/gist_id}","starred_url":"https://api.github.com/users/virtuman/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/virtuman/subscriptions","organizations_url":"https://api.github.com/users/virtuman/orgs","repos_url":"https://api.github.com/users/virtuman/repos","events_url":"https://api.github.com/users/virtuman/events{/privacy}","received_events_url":"https://api.github.com/users/virtuman/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2015-04-12T18:22:49Z","updated_at":"2015-04-13T23:20:35Z","closed_at":"2015-04-13T23:20:35Z","body":"Trying to use MongoDB as database, getting exceptions.\r\nLooked through the docs, didn't find any version limitations but as of right now have the following versions installed:\r\nCENTOS 6.x\r\nmongodb 3.0.2\r\n\r\n```\r\npip2.7 freeze\r\namqp==1.4.6\r\nastroid==1.3.6\r\nbackports.ssl-match-hostname==3.4.0.2\r\nBeautifulSoup==3.2.1\r\ncertifi==14.5.14\r\ncffi==0.9.2\r\ncharacteristic==14.3.0\r\nchardet==2.3.0\r\nclick==4.0\r\ncryptography==0.8.1\r\ncssselect==0.9.1\r\nelasticsearch==1.4.0\r\nenum34==1.0.4\r\nFlask==0.10.1\r\nFlask-Login==0.2.11\r\nipdb==0.8\r\nipython==3.1.0\r\nitsdangerous==0.24\r\nJinja2==2.7.3\r\nlinecache2==1.0.0\r\nlogilab-common==0.63.2\r\nlxml==3.4.2\r\nMarkupSafe==0.23\r\nmeld3==1.0.0\r\nmysql-connector-python==2.0.3\r\npika==0.9.14\r\npyasn1==0.1.7\r\npyasn1-modules==0.0.5\r\npycparser==2.10\r\npycurl==7.19.5.1\r\npymongo==3.0\r\npyOpenSSL==0.14\r\npyquery==1.2.9\r\npyspider==0.3.3\r\npython-dateutil==2.4.1\r\nqueuelib==1.2.2\r\nrequests==2.6.0\r\nScrapy==0.24.5\r\nservice-identity==14.0.0\r\nsix==1.9.0\r\nSQLAlchemy==0.9.9\r\nsupervisor==3.1.3\r\ntornado==4.1\r\ntraceback2==1.4.0\r\nTwisted==15.0.0\r\nu-msgpack-python==2.0\r\nunittest2==1.0.1\r\nurllib3==1.10.2\r\nw3lib==1.11.0\r\nWerkzeug==0.10.4\r\nzope.interface==4.1.2\r\n```\r\n\r\npyspider 0-3-3 installs nicely but when attempt to run it get the following error:\r\n\r\n```\r\n[I 150412 14:17:31 processor:162] processor starting...\r\nException AttributeError: \"'Cursor' object has no attribute '_Cursor__id'\" in <bound method Cursor.__del__ of <pymongo.cursor.Cursor object at 0x7fcb4c4bd890>> ignored\r\n[E 150412 14:17:31 scheduler:405] __init__() got an unexpected keyword argument 'fields'\r\n    Traceback (most recent call last):\r\n      File \"/usr/local/lib/python2.7/site-packages/pyspider/scheduler/scheduler.py\", line 400, in run\r\n        self.run_once()\r\n      File \"/usr/local/lib/python2.7/site-packages/pyspider/scheduler/scheduler.py\", line 384, in run_once\r\n        self._update_projects()\r\n      File \"/usr/local/lib/python2.7/site-packages/pyspider/scheduler/scheduler.py\", line 81, in _update_projects\r\n        for project in self.projectdb.check_update(self._last_update_project):\r\n      File \"/usr/local/lib/python2.7/site-packages/pyspider/database/mongodb/projectdb.py\", line 50, in check_update\r\n        for project in self.get_all(fields=('updatetime', 'name')):\r\n      File \"/usr/local/lib/python2.7/site-packages/pyspider/database/mongodb/projectdb.py\", line 37, in get_all\r\n        for each in self.collection.find({}, fields=fields):\r\n      File \"/usr/local/lib/python2.7/site-packages/pymongo/collection.py\", line 924, in find\r\n        return Cursor(self, *args, **kwargs)\r\n    TypeError: __init__() got an unexpected keyword argument 'fields'\r\n[I 150412 14:17:31 app:65] webui running on http://0.0.0.0:5000/\r\n```\r\n\r\naccording to pymongo - second arg in collection.find() is something that they refer to as Projections, but appears to be a simple json-like object:\r\n\r\nhttp://docs.mongodb.org/manual/reference/method/db.collection.find/\r\n\r\nis current version of pyspider supposed to (tested) work with mongodb at all?\r\n\r\nThank you"},"comment":{"url":"https://api.github.com/repos/binux/pyspider/issues/comments/92531649","html_url":"https://github.com/binux/pyspider/issues/178#issuecomment-92531649","issue_url":"https://api.github.com/repos/binux/pyspider/issues/178","id":92531649,"user":{"login":"virtuman","id":1087996,"avatar_url":"https://avatars.githubusercontent.com/u/1087996?v=3","gravatar_id":"","url":"https://api.github.com/users/virtuman","html_url":"https://github.com/virtuman","followers_url":"https://api.github.com/users/virtuman/followers","following_url":"https://api.github.com/users/virtuman/following{/other_user}","gists_url":"https://api.github.com/users/virtuman/gists{/gist_id}","starred_url":"https://api.github.com/users/virtuman/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/virtuman/subscriptions","organizations_url":"https://api.github.com/users/virtuman/orgs","repos_url":"https://api.github.com/users/virtuman/repos","events_url":"https://api.github.com/users/virtuman/events{/privacy}","received_events_url":"https://api.github.com/users/virtuman/received_events","type":"User","site_admin":false},"created_at":"2015-04-13T23:20:35Z","updated_at":"2015-04-13T23:20:35Z","body":"thank you"}}