{"action":"created","issue":{"url":"https://api.github.com/repos/prerender/prerender/issues/95","labels_url":"https://api.github.com/repos/prerender/prerender/issues/95/labels{/name}","comments_url":"https://api.github.com/repos/prerender/prerender/issues/95/comments","events_url":"https://api.github.com/repos/prerender/prerender/issues/95/events","html_url":"https://github.com/prerender/prerender/issues/95","id":35538141,"number":95,"title":"Server doesn't fully recover after a PhantomJS crash","user":{"login":"mbenford","id":446867,"avatar_url":"https://avatars.githubusercontent.com/u/446867?v=3","gravatar_id":"","url":"https://api.github.com/users/mbenford","html_url":"https://github.com/mbenford","followers_url":"https://api.github.com/users/mbenford/followers","following_url":"https://api.github.com/users/mbenford/following{/other_user}","gists_url":"https://api.github.com/users/mbenford/gists{/gist_id}","starred_url":"https://api.github.com/users/mbenford/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mbenford/subscriptions","organizations_url":"https://api.github.com/users/mbenford/orgs","repos_url":"https://api.github.com/users/mbenford/repos","events_url":"https://api.github.com/users/mbenford/events{/privacy}","received_events_url":"https://api.github.com/users/mbenford/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/prerender/prerender/labels/in%20progress","name":"in progress","color":"ededed"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":64,"created_at":"2014-06-11T23:31:18Z","updated_at":"2015-11-04T18:38:31Z","closed_at":null,"body":"Hi. First off, congrats on this amazing project! :+1: \r\n\r\nWe've been evaluating Prerender as an alternative to replace our in-house solution for handling AJAX crawling (which is, by the way, similar to yours to a degree), and we've been experiencing some issues, mostly related to PhantomJS, as far as I can tell.\r\n\r\nThe first issue regards the number of PhantomJS workers. We've figured out that any number greater than 1 causes the server to hang after a small amount of requests. It seems that [other people](http://www.jonben.net/prerender-io-the-highs-and-lows/) have experienced that as well.\r\n\r\nAnd the second, more critical issue is related to the PhantomJS restarting process. After tweaking the timeouts and setting the `PHANTOM_CLUSTER_NUM_WORKERS` to 1, we've managed to make the server pretty stable for some requests, but that stability disappears as soon as PhantonJS crashes or is killed by Prerender. The following graph illustrates that:\r\n\r\n![prerender](https://cloud.githubusercontent.com/assets/446867/3250584/9bff339e-f1a7-11e3-91b0-6c80d2bd10e4.jpg)\r\n\r\nThe server seems to never fully recover after a PhantomJS crash. My best guess is that when a request arrives while PhantomJS is still starting up, bad things happen.\r\n\r\nI'm willing to debug Prerender's source code in order to fix or at least understand better why the second issue is happening, and I'm looking for some guidance on where to begin. I appreciate any help.\r\n\r\nFor what it's worth, we're running Prerender on a dedicated Ubuntu machine and using JMeter to generate one request per second, which is how often Googlebot hits our servers on average."},"comment":{"url":"https://api.github.com/repos/prerender/prerender/issues/comments/153823605","html_url":"https://github.com/prerender/prerender/issues/95#issuecomment-153823605","issue_url":"https://api.github.com/repos/prerender/prerender/issues/95","id":153823605,"user":{"login":"thoop","id":1259374,"avatar_url":"https://avatars.githubusercontent.com/u/1259374?v=3","gravatar_id":"","url":"https://api.github.com/users/thoop","html_url":"https://github.com/thoop","followers_url":"https://api.github.com/users/thoop/followers","following_url":"https://api.github.com/users/thoop/following{/other_user}","gists_url":"https://api.github.com/users/thoop/gists{/gist_id}","starred_url":"https://api.github.com/users/thoop/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/thoop/subscriptions","organizations_url":"https://api.github.com/users/thoop/orgs","repos_url":"https://api.github.com/users/thoop/repos","events_url":"https://api.github.com/users/thoop/events{/privacy}","received_events_url":"https://api.github.com/users/thoop/received_events","type":"User","site_admin":false},"created_at":"2015-11-04T18:38:31Z","updated_at":"2015-11-04T18:38:31Z","body":"Yeah it looks like phantom-html-to-pdf tries to start quite a few phantomjs servers on the same machine, so you might run into issues if you don't have a really beefy server. The same can happen with Prerender."}}