{"action":"created","issue":{"url":"https://api.github.com/repos/medialab/hyphe/issues/135","labels_url":"https://api.github.com/repos/medialab/hyphe/issues/135/labels{/name}","comments_url":"https://api.github.com/repos/medialab/hyphe/issues/135/comments","events_url":"https://api.github.com/repos/medialab/hyphe/issues/135/events","html_url":"https://github.com/medialab/hyphe/pull/135","id":85615253,"number":135,"title":"[WIP] Docker install","user":{"login":"oncletom","id":138627,"avatar_url":"https://avatars.githubusercontent.com/u/138627?v=3","gravatar_id":"","url":"https://api.github.com/users/oncletom","html_url":"https://github.com/oncletom","followers_url":"https://api.github.com/users/oncletom/followers","following_url":"https://api.github.com/users/oncletom/following{/other_user}","gists_url":"https://api.github.com/users/oncletom/gists{/gist_id}","starred_url":"https://api.github.com/users/oncletom/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/oncletom/subscriptions","organizations_url":"https://api.github.com/users/oncletom/orgs","repos_url":"https://api.github.com/users/oncletom/repos","events_url":"https://api.github.com/users/oncletom/events{/privacy}","received_events_url":"https://api.github.com/users/oncletom/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":7,"created_at":"2015-06-05T19:29:43Z","updated_at":"2015-06-10T10:54:15Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/medialab/hyphe/pulls/135","html_url":"https://github.com/medialab/hyphe/pull/135","diff_url":"https://github.com/medialab/hyphe/pull/135.diff","patch_url":"https://github.com/medialab/hyphe/pull/135.patch"},"body":"Some undergoing work to only pull stuff from the cloud/NSA tap surveillance network to analyse the network:\n- env variables over configuration files for some touchy points (easier to provision from a CLI)\n- each service is more isolated\n- each service carries only its required dependencies\n- each service is easier to upgrade because they don't rely on a global stack, possibly conflictual\n- each service install is now documented in a way you can guarantee it will work\n- Docker compose to orchestrate services altogether\n\n```bash\ndocker-compose up\n```\n\nAnd we are done."},"comment":{"url":"https://api.github.com/repos/medialab/hyphe/issues/comments/110699080","html_url":"https://github.com/medialab/hyphe/pull/135#issuecomment-110699080","issue_url":"https://api.github.com/repos/medialab/hyphe/issues/135","id":110699080,"user":{"login":"oncletom","id":138627,"avatar_url":"https://avatars.githubusercontent.com/u/138627?v=3","gravatar_id":"","url":"https://api.github.com/users/oncletom","html_url":"https://github.com/oncletom","followers_url":"https://api.github.com/users/oncletom/followers","following_url":"https://api.github.com/users/oncletom/following{/other_user}","gists_url":"https://api.github.com/users/oncletom/gists{/gist_id}","starred_url":"https://api.github.com/users/oncletom/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/oncletom/subscriptions","organizations_url":"https://api.github.com/users/oncletom/orgs","repos_url":"https://api.github.com/users/oncletom/repos","events_url":"https://api.github.com/users/oncletom/events{/privacy}","received_events_url":"https://api.github.com/users/oncletom/received_events","type":"User","site_admin":false},"created_at":"2015-06-10T10:54:15Z","updated_at":"2015-06-10T10:54:15Z","body":"@boogheta here is the output of the commands:\r\n\r\n```\r\nroot@de161d7f7f8b:/# mongo hyphe --eval \"db.test.pages.count()\"\r\nMongoDB shell version: 3.0.3\r\nconnecting to: hyphe\r\n0\r\nroot@de161d7f7f8b:/# mongo hyphe --eval \"db.test.queue.count()\"\r\nMongoDB shell version: 3.0.3\r\nconnecting to: hyphe\r\n0\r\nroot@de161d7f7f8b:/# mongo hyphe --eval \"db.test.jobs.forEach(printjson)\"\r\nMongoDB shell version: 3.0.3\r\nconnecting to: hyphe\r\n2015-06-10T10:44:09.843+0000 E QUERY    TypeError: Property 'forEach' of object hyphe.test.jobs is not a function\r\n    at (shell eval):1:14\r\n```\r\n\r\nFurthermore:\r\n\r\n```\r\n$ mongo\r\n> show dbs\r\nhyphe  0.078GB\r\nlocal  0.078GB\r\n> use hyphe\r\nswitched to db hyphe\r\n> show collections\r\ncorpus\r\nsystem.indexes\r\ntest.jobs\r\ntest.logs\r\ntest.pages\r\ntest.queries\r\ntest.queue\r\ntest.stats\r\n> db.test.pages.find()\r\n```\r\n\r\nNothing is stored in pages apparently. Maybe the Mongodb connection is failing within the egg.\r\n\r\nEDIT: that's it. I unzipped the uploaded egg and the MONGO_HOST value is set to localhost:\r\n\r\n```bash\r\nMONGO_HOST = 'localhost'\r\nMONGO_PORT = 27017\r\nMONGO_DB = 'hyphe'\r\nMONGO_QUEUE_COL = 'test.queue'\r\nMONGO_PAGESTORE_COL = 'test.pages'\r\n```\r\n\r\nAre the settings of `PHANTOM` used? Because it is set to the following:\r\n\r\n```json\r\nPHANTOM = {\r\n  \"PATH\": os.path.join('/app', 'bin', 'hyphe-phantomjs-2.0.0'),\r\n  \"JS_PATH\": os.path.join('/app', 'hyphe_backend', 'crawler', BOT_NAME, 'spiders', 'js'),\r\n  \"TIMEOUT\": 600,\r\n  \"IDLE_TIMEOUT\": 20,\r\n  \"AJAX_TIMEOUT\": 15\r\n}\r\n```\r\n\r\nAnd they will definitely fail as the crawler is deployed in a separate container."}}