{"action":"created","issue":{"url":"https://api.github.com/repos/jazeee/jazeee-meteor-spiderable/issues/14","labels_url":"https://api.github.com/repos/jazeee/jazeee-meteor-spiderable/issues/14/labels{/name}","comments_url":"https://api.github.com/repos/jazeee/jazeee-meteor-spiderable/issues/14/comments","events_url":"https://api.github.com/repos/jazeee/jazeee-meteor-spiderable/issues/14/events","html_url":"https://github.com/jazeee/jazeee-meteor-spiderable/issues/14","id":98768177,"number":14,"title":"204 - No content","user":{"login":"Buom01","id":1537208,"avatar_url":"https://avatars.githubusercontent.com/u/1537208?v=3","gravatar_id":"","url":"https://api.github.com/users/Buom01","html_url":"https://github.com/Buom01","followers_url":"https://api.github.com/users/Buom01/followers","following_url":"https://api.github.com/users/Buom01/following{/other_user}","gists_url":"https://api.github.com/users/Buom01/gists{/gist_id}","starred_url":"https://api.github.com/users/Buom01/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Buom01/subscriptions","organizations_url":"https://api.github.com/users/Buom01/orgs","repos_url":"https://api.github.com/users/Buom01/repos","events_url":"https://api.github.com/users/Buom01/events{/privacy}","received_events_url":"https://api.github.com/users/Buom01/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":6,"created_at":"2015-08-03T15:27:37Z","updated_at":"2015-08-10T15:47:22Z","closed_at":null,"body":"## Hi,\r\nI have a probleme with my website, what I try to do, my pages return nothing with a 204 HTTP status code with `?_escaped_fragment_=`, else, the 404 page send correctly the 404 HTTP status code, but is also empty\r\n\r\n## Notes:\r\n- My host is openshift, I have a free plan\r\n- My host has an Apache proxy for routing <-- Can be the probleme ?\r\n- I use demeteorizer to convert to simple nodejs app\r\n- I used npm to install phantomjs\r\n\r\n## Code:\r\n### spiderable-test.js\r\n```javascript\r\nif(Meteor.isServer){\r\n  Spiderable.debug = true;\r\n  Spiderable.customQuery = true;\r\n}\r\nRouter.configure({\r\n  notFoundTemplate: '_404'\r\n});\r\nRouter.plugin('dataNotFound', {notFoundTemplate: '_404'});\r\n\r\nRouter.route('/', function () {\r\n  this.render('main');\r\n});\r\n\r\nRouter.onAfterAction(function(){\r\n  if(this.ready()){ \r\n    Meteor.isReadyForSpiderable = true;\r\n  }\r\n});\r\n\r\nconsole.log('Scripting...');\r\n```\r\n### spiderable-test.html\r\n```handlebars\r\n<head>\r\n  <title>spiderable-test</title>\r\n</head>\r\n\r\n<body>\r\n</body>\r\n\r\n<template name=\"main\">\r\n  <h1>Page d'accueil</h1>\r\n</template>\r\n<template name=\"_404\">\r\n  <h1>404</h1>\r\n  <h3>Oops, page not found</h3>\r\n  <p>Sorry, page you're requested is not exists or was deleted</p>\r\n</template>\r\n```\r\n\r\n## Mongo content:\r\n### Complete informations\r\n```shell\r\n[test-buom01.rhcloud.com ******]\\> mongo $OPENSHIFT_MONGODB_URLtest\r\nMongoDB shell version: 2.4.9                                                                                                                                                                               \r\nconnecting to: 127.*.*.*:27017/admin                                                                                                                                                                  \r\nWelcome to the MongoDB shell.                                                                                                                                                                              \r\nFor interactive help, type \"help\".                                                                                                                                                                         \r\nFor more comprehensive documentation, see                                                                                                                                                                  \r\n        http://docs.mongodb.org/\r\nQuestions? Try the support group\r\n        http://groups.google.com/group/mongodb-user\r\n> use test\r\nswitched to db test\r\n> db.SpiderableCacheCollection.find({})\r\n{ \"_id\" : \"yLhw4TicXDzEeT8E5\", \"hash\" : \"f807c97ecbb6087b674f75cc45e714675b8cdd0b56959f7600c89ebc860f9678\", \"url\" : \"http://test-buom01.rhcloud.com/?___isRunningPhantomJS___=true\", \"headers\" : [      { \"name\" : \"Date\",         \"value\" : \"Mon, 03 Aug 2015 13:41:14 GMT\" },    {       \"name\" : \"Vary\",        \"value\" : \"Accept-Encoding\" },  {       \"name\" : \"Content-Type\",        \"value\" : \"text/html; charset=utf-8\" },    {       \"name\" : \"Content-Encoding\",    \"value\" : \"gzip\" },     {       \"name\" : \"Keep-Alive\",  \"value\" : \"timeout=15, max=100\" },      {       \"name\" : \"Connection\",  \"value\" : \"Keep-Alive\" } ], \"content\" : \"<!DOCTYPE html><html><head>\\n  <link rel=\\\"stylesheet\\\" type=\\\"text/css\\\" class=\\\"__meteor-css__\\\" href=\\\"/20ae2c8d51b2507244e598844414ecdec2615ce3.css\\\">\\n\\n\\n\\n\\n  \\n\\n\\n\\n\\n<title>spiderable-test</title>\\n</head>\\n<body>\\n\\n\\n\\n<h1>Page d'accueil</h1></body></html>\", \"status\" : 204, \"createdAt\" : ISODate(\"2015-08-03T13:41:15.360Z\") }\r\n> \r\n```\r\n\r\n### Just its content\r\n```json\r\n{\r\n   \"_id\":\"yLhw4TicXDzEeT8E5\",\r\n   \"hash\":\"f807c97ecbb6087b674f75cc45e714675b8cdd0b56959f7600c89ebc860f9678\",\r\n   \"url\":\"http://test-buom01.rhcloud.com/?___isRunningPhantomJS___=true\",\r\n   \"headers\":[\r\n      {\r\n         \"name\":\"Date\",\r\n         \"value\":\"Mon, 03 Aug 2015 13:41:14 GMT\"\r\n      },\r\n      {\r\n         \"name\":\"Vary\",\r\n         \"value\":\"Accept-Encoding\"\r\n      },\r\n      {\r\n         \"name\":\"Content-Type\",\r\n         \"value\":\"text/html; charset=utf-8\"\r\n      },\r\n      {\r\n         \"name\":\"Content-Encoding\",\r\n         \"value\":\"gzip\"\r\n      },\r\n      {\r\n         \"name\":\"Keep-Alive\",\r\n         \"value\":\"timeout=15, max=100\"\r\n      },\r\n      {\r\n         \"name\":\"Connection\",\r\n         \"value\":\"Keep-Alive\"\r\n      }\r\n   ],\r\n   \"content\":\"<!DOCTYPE html><html><head>\\n  <link rel=\\\"stylesheet\\\" type=\\\"text/css\\\" class=\\\"__meteor-css__\\\" href=\\\"/20ae2c8d51b2507244e598844414ecdec2615ce3.css\\\">\\n\\n\\n\\n\\n  \\n\\n\\n\\n\\n<title>spiderable-test</title>\\n</head>\\n<body>\\n\\n\\n\\n<h1>Page d'accueil</h1></body></html>\",\r\n   \"status\":204,\r\n   \"createdAt\":   ISODate(\"2015-08-03T13:41:15.360   Z\")\r\n}\r\n```\r\n\r\n## My building home made script:\r\n### .build2.sh\r\n```sh\r\n./.compress2.sh\r\nexport MINI=\"./.mini/nodejs\"\r\necho \"{{{{BUILDING}}}}\"\r\ncd $MINI\r\necho \"Remove output\"\r\nchmod -R 777 ./.demeteorized\r\nrm -R ./.demetorized\r\necho \"Build and demeteorization...\"\r\ndemeteorizer\r\necho \"Fix permissions again...\"\r\nchmod -R 777 ./.demeteorized\r\n\r\ncd ./.demeteorized\r\n\r\necho \"Adding env vars...\"\r\n\r\n# settings.json is actually empty\r\necho \"process.env.METEOR_SETTINGS = '$(echo $(cat ../settings.json))';\r\n$(cat ./main.js)\" > ./main.js \r\n\r\n\r\n#sed -i '1i process.env.MAIL_URL = \"smtp://*********/\";' ./main.js\r\n#sed -i '1i process.env.ROOT_URL = (\"http://\" + process.env.OPENSHIFT_APP_DNS) || \"http://localhost:8000\"' ./main.js\r\n#sed -i '1i process.env.MONGO_URL = (process.env.OPENSHIFT_MONGODB_DB_URL + process.env.OPENSHIFT_APP_NAME) || \"mongodb://localhost:27017/meteor\";' ./main.js\r\n#sed -i '1i process.env.PORT = process.env.OPENSHIFT_NODEJS_PORT || 8000;' ./main.js\r\n#sed -i '1i process.env.BIND_IP = process.env.OPENSHIFT_NODEJS_IP || \"127.0.0.1\";' ./main.js\r\n\r\n\r\n\r\n#sed -i '1i process.env.MAIL_URL = \"smtp://*******/\";' ./main.js\r\nsed -i '1i process.env.ROOT_URL = \"http://\" + (process.env.OPENSHIFT_APP_DNS || \"localhost:8000\");' ./main.js\r\n#sed -i '1i process.env.ROOT_URL = \"http://\"+ process.env.OPENSHIFT_NODEJS_IP + \":\" + process.env.OPENSHIFT_NODEJS_PORT;' ./main.js\r\nsed -i '1i process.env.MONGO_URL = (process.env.OPENSHIFT_MONGODB_DB_URL + process.env.OPENSHIFT_APP_NAME) || \"mongodb://localhost:27017/meteor\";' ./main.js\r\nsed -i '1i process.env.PORT = process.env.OPENSHIFT_NODEJS_PORT || 8000;' ./main.js\r\nsed -i '1i process.env.BIND_IP = process.env.OPENSHIFT_NODEJS_IP || \"127.0.0.1\";' ./main.js\r\n\r\n\r\necho \"Copying into git directory...\"\r\ncd ../../..\r\nrm -R ./.end/*\r\n#mkdir ./.end\r\n#cp -R ./.static-files/.git ./.end/\r\n#cp -R ./.static-files/.openshift ./.end/\r\ncp -R $MINI/.demeteorized/* ./.end/\r\necho \"Done\"\r\n```\r\n\r\n### .compress2.sh\r\n```sh\r\necho \"{{{{COMPRESSION}}}}\"\r\nexport MINI=\"./.mini/nodejs\"\r\necho \"Fix permissions...\"\r\nchmod -R 777 $MINI\r\necho \"Removing...\"\r\nrm -R $MINI\r\necho \"Copying...\"\r\nmkdir $MINI\r\ncp -R * $MINI\r\ncp -R ./.meteor $MINI\r\necho \"Minifying...\"\r\n#htmlminify -o $MINI/spiderable-test.html $MINI/spiderable-test.html\r\necho \"Remove enters...\"\r\n#echo  $(cat $MINI/spiderable-test.html)>$MINI/spiderable-test.html\r\necho \"Done\"\r\n```\r\n\r\n### .deploy2.sh\r\n```sh\r\ncd .end\r\ngit add .\r\ngit commit -m \"Update\"\r\ngit push\r\n```\r\n## Packages\r\n- gadicohen:phantomjs               0.0.2  phantomjs installation script; wrapper for node phantomjs package + Sun\r\n- iron:router                       1.0.9  Routing specifically designed for Meteor\r\n- jazeee:spiderable-longer-timeout  1.2.5  Extended spiderable package: SSL, caching, longer timeout, no stdin issues, publish flag\r\n- meteor-platform                   1.2.2  Include a standard set of Meteor packages in your app\r\n\r\n## Se it yourselft\r\nhttp://test-buom01.rhcloud.com/\r\nhttp://test-buom01.rhcloud.com/?_escaped_fragment_=\r\n\r\n## Conclusion\r\nAnybody can help me ?\r\nWhat can I do ?\r\n\r\n---\r\n\r\nThank you for reading, and sorry if I have a bad English"},"comment":{"url":"https://api.github.com/repos/jazeee/jazeee-meteor-spiderable/issues/comments/129502110","html_url":"https://github.com/jazeee/jazeee-meteor-spiderable/issues/14#issuecomment-129502110","issue_url":"https://api.github.com/repos/jazeee/jazeee-meteor-spiderable/issues/14","id":129502110,"user":{"login":"dr-dimitru","id":1686778,"avatar_url":"https://avatars.githubusercontent.com/u/1686778?v=3","gravatar_id":"","url":"https://api.github.com/users/dr-dimitru","html_url":"https://github.com/dr-dimitru","followers_url":"https://api.github.com/users/dr-dimitru/followers","following_url":"https://api.github.com/users/dr-dimitru/following{/other_user}","gists_url":"https://api.github.com/users/dr-dimitru/gists{/gist_id}","starred_url":"https://api.github.com/users/dr-dimitru/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dr-dimitru/subscriptions","organizations_url":"https://api.github.com/users/dr-dimitru/orgs","repos_url":"https://api.github.com/users/dr-dimitru/repos","events_url":"https://api.github.com/users/dr-dimitru/events{/privacy}","received_events_url":"https://api.github.com/users/dr-dimitru/received_events","type":"User","site_admin":false},"created_at":"2015-08-10T15:47:22Z","updated_at":"2015-08-10T15:47:22Z","body":"Without this logic statement you will not receive status code on redirects\n\n\n\n> On 10 Aug 2015, at 18:40, Jaz <notifications@github.com> wrote:\n> \n> I found that this issue does exist, and is due to a bug in the phantomjs script.\n> \n> It appears to depend on whether one uses a proxy, or similar. For example, if one uses nginx in front of meteor, for SSL services, for example.\n> \n> One way to test is, create a meteor server, with nginx wrapper for SSL. Then, test the phantomjs script:\n> \n> phantomjs --load-images=no --ssl-protocol=TLSv1 --ignore-ssl-errors=true --web-security=false jazeee-meteor-spiderable/lib/phantom_script.js https://your-server\n> This returns a 204. The reason is that https://github.com/jazeee/jazeee-meteor-spiderable/blob/master/lib/phantom_script.js#L60 is not correct.\n> \n> page.onResourceReceived(...) triggers on any resource request, including image assets or Meteor Websocket connection. In the first part of that function, we correctly check for the URL, however, for the remaining part, we accept any status. Since this seems unnecessary, I am removing that part to fix this bug.\n> \n> —\n> Reply to this email directly or view it on GitHub.\n> \n"}}