{"action":"created","issue":{"url":"https://api.github.com/repos/nodejs/node/issues/3370","labels_url":"https://api.github.com/repos/nodejs/node/issues/3370/labels{/name}","comments_url":"https://api.github.com/repos/nodejs/node/issues/3370/comments","events_url":"https://api.github.com/repos/nodejs/node/issues/3370/events","html_url":"https://github.com/nodejs/node/issues/3370","id":111436867,"number":3370,"title":"Node 4.2 large memory spikes and timeouts on Heroku ","user":{"login":"AndrewBarba","id":1316152,"avatar_url":"https://avatars.githubusercontent.com/u/1316152?v=3","gravatar_id":"","url":"https://api.github.com/users/AndrewBarba","html_url":"https://github.com/AndrewBarba","followers_url":"https://api.github.com/users/AndrewBarba/followers","following_url":"https://api.github.com/users/AndrewBarba/following{/other_user}","gists_url":"https://api.github.com/users/AndrewBarba/gists{/gist_id}","starred_url":"https://api.github.com/users/AndrewBarba/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AndrewBarba/subscriptions","organizations_url":"https://api.github.com/users/AndrewBarba/orgs","repos_url":"https://api.github.com/users/AndrewBarba/repos","events_url":"https://api.github.com/users/AndrewBarba/events{/privacy}","received_events_url":"https://api.github.com/users/AndrewBarba/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/nodejs/node/labels/memory","name":"memory","color":"eb6420"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":26,"created_at":"2015-10-14T16:29:36Z","updated_at":"2015-11-19T17:30:06Z","closed_at":null,"body":"Below is a screenshot of the past 12 hours which has been a total disaster for us to say the least. After updating to node 4.2 (from 0.10) in production we immediately exceeded all memory quotas and experienced a high volume of timeouts (even with no load and memory under the 1GB limit). \r\n\r\nFirst, I apologize if this is not the place for this. I am happy to move discussion somewhere else and we will help diagnose whatever you guys need. We did this same parade with Node 0.12 and had to downgrade to 0.10.\r\n\r\nSecond, and I guess the real question here, is Heroku's 512MB of ram simply not enough to run Node 4.x? If that is the case, cool, but memory constraints definitely need to be made more clear. \r\n\r\nTimeline:\r\n- Tue, Oct. 13th, at 2pm EST we deployed Node 4.2.0 with cluster enabled running 2 threads. Immediately hit 512 memory limit as seen in the picture below.\r\n- Tue, Oct. 13th, at 2:15pm EST we removed cluster completely. Hit memory limits 30 min later.\r\n- Tue, Oct. 13th, at 4:00pm EST we saw Node 4.2.1 was released, deployed 4.2.1. Continued to hit memory limits\r\n- Tue, Oct. 13th, at 6:00pm EST we doubled memory to 1GB\r\n\r\nIn general you can see the memory is all over the place, maybe that is expected with newer versions of V8...\r\n\r\n<img width=\"1162\" alt=\"screen shot 2015-10-14 at 12 08 03 pm\" src=\"https://cloud.githubusercontent.com/assets/1316152/10490184/a34f5bae-726d-11e5-8c1c-cf707acc3725.png\">\r\n\r\nAlthough I don't have a screen, you can see in the first part of the graph running Node 0.10 that it stays almost perfectly flat at 256MB of ram. Under any load, that was consistent.\r\n\r\nFor reference, here is a load test we did in a dev environment running Node 4.2.1, cluster forked to 4 processes, and about 5k rpms. Also immediately hit the higher 1GB memory limit. We then dropped this down to 2 forked processes with the same result.  \r\n\r\n<img width=\"1161\" alt=\"screen shot 2015-10-14 at 12 26 15 pm\" src=\"https://cloud.githubusercontent.com/assets/1316152/10490445/0d76593c-726f-11e5-8199-b8585592fc7c.png\">\r\n"},"comment":{"url":"https://api.github.com/repos/nodejs/node/issues/comments/158128655","html_url":"https://github.com/nodejs/node/issues/3370#issuecomment-158128655","issue_url":"https://api.github.com/repos/nodejs/node/issues/3370","id":158128655,"user":{"login":"piranna","id":532414,"avatar_url":"https://avatars.githubusercontent.com/u/532414?v=3","gravatar_id":"","url":"https://api.github.com/users/piranna","html_url":"https://github.com/piranna","followers_url":"https://api.github.com/users/piranna/followers","following_url":"https://api.github.com/users/piranna/following{/other_user}","gists_url":"https://api.github.com/users/piranna/gists{/gist_id}","starred_url":"https://api.github.com/users/piranna/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/piranna/subscriptions","organizations_url":"https://api.github.com/users/piranna/orgs","repos_url":"https://api.github.com/users/piranna/repos","events_url":"https://api.github.com/users/piranna/events{/privacy}","received_events_url":"https://api.github.com/users/piranna/received_events","type":"User","site_admin":false},"created_at":"2015-11-19T17:30:06Z","updated_at":"2015-11-19T17:30:06Z","body":"> @piranna What exact exception do you mean?\r\n\r\nThe error I get is an Out of Memory exception, thrown at process level:\r\n![pantallazo-qemu](https://cloud.githubusercontent.com/assets/532414/11278335/be7d6760-8eea-11e5-99e5-e5980372bf9d.png)\r\nIt happens when calling `malloc()` and the system don't have enough free memory or a contiguos free chunk big enough for the requested size. I propose that when creating a new Javascript object, when trying to reserve memory capture the exception on the C++ area and if so, exec the garbage colector and try to reserve it again. Probably this is more a v8 issue..."}}