{"action":"created","issue":{"url":"https://api.github.com/repos/lovell/sharp/issues/225","labels_url":"https://api.github.com/repos/lovell/sharp/issues/225/labels{/name}","comments_url":"https://api.github.com/repos/lovell/sharp/issues/225/comments","events_url":"https://api.github.com/repos/lovell/sharp/issues/225/events","html_url":"https://github.com/lovell/sharp/issues/225","id":81720962,"number":225,"title":"Heavy load with large GIFs eventually make the app unresponsive without an error","user":{"login":"Robert-Wett","id":1927957,"avatar_url":"https://avatars.githubusercontent.com/u/1927957?v=3","gravatar_id":"","url":"https://api.github.com/users/Robert-Wett","html_url":"https://github.com/Robert-Wett","followers_url":"https://api.github.com/users/Robert-Wett/followers","following_url":"https://api.github.com/users/Robert-Wett/following{/other_user}","gists_url":"https://api.github.com/users/Robert-Wett/gists{/gist_id}","starred_url":"https://api.github.com/users/Robert-Wett/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Robert-Wett/subscriptions","organizations_url":"https://api.github.com/users/Robert-Wett/orgs","repos_url":"https://api.github.com/users/Robert-Wett/repos","events_url":"https://api.github.com/users/Robert-Wett/events{/privacy}","received_events_url":"https://api.github.com/users/Robert-Wett/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/lovell/sharp/labels/question","name":"question","color":"fef2c0"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2015-05-28T02:24:43Z","updated_at":"2015-06-07T05:07:33Z","closed_at":null,"body":"I've been using your library to power all of our sites thumbnailing/image resizing operations, and have been running into a lot of issues lately under heavy load, specifically with GIFs. We are running `^0.9.1` sharp, and using `vips-8.0.1` for the GIF functionality.\r\n\r\nThe issue is that when a server using sharp has 5+ or so large (9+ MB, sharp.cache set to 0, but it exhibited the same behavior with defaults ) GIF's it needs to process, it will at some point (sometimes, not always) just become unresponsive and no longer respond to any requests without throwing any kind of error. I am having to `.toBuffer` these jobs before I call `.toFile`, but not sure if that matters if I'm not using streams. The worst part is that it cripples all functionality, but the box is still somehow able to maintain it's consumer connection to rabbitmq and will still pull messages and hoard them until manually restarted (that part just sucks, not expecting you to know why that happens :smile: ). We are in the process of setting up load testing environments so that we can play around with `sharp.concurrency` and `UV_THREADPOOL_SIZE` to see if we can eliminate this behavior as a possibility, but man would it be nice if there were a way to throw an error when it won't be able to read in a large GIF/file to memory (or w/e is crashing it) so I could reject the job and send it off to another box that has more memory available.  \r\n\r\nThanks for your work!"},"comment":{"url":"https://api.github.com/repos/lovell/sharp/issues/comments/109694356","html_url":"https://github.com/lovell/sharp/issues/225#issuecomment-109694356","issue_url":"https://api.github.com/repos/lovell/sharp/issues/225","id":109694356,"user":{"login":"Robert-Wett","id":1927957,"avatar_url":"https://avatars.githubusercontent.com/u/1927957?v=3","gravatar_id":"","url":"https://api.github.com/users/Robert-Wett","html_url":"https://github.com/Robert-Wett","followers_url":"https://api.github.com/users/Robert-Wett/followers","following_url":"https://api.github.com/users/Robert-Wett/following{/other_user}","gists_url":"https://api.github.com/users/Robert-Wett/gists{/gist_id}","starred_url":"https://api.github.com/users/Robert-Wett/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Robert-Wett/subscriptions","organizations_url":"https://api.github.com/users/Robert-Wett/orgs","repos_url":"https://api.github.com/users/Robert-Wett/repos","events_url":"https://api.github.com/users/Robert-Wett/events{/privacy}","received_events_url":"https://api.github.com/users/Robert-Wett/received_events","type":"User","site_admin":false},"created_at":"2015-06-07T05:07:33Z","updated_at":"2015-06-07T05:07:33Z","body":"Hey @lovell! Thanks for being so responsive.\r\nUnfortunately, cursory experiments with the usual settings didn't seem to yield any better results - it's really easy to run into an OOM error with GIFs. I haven't yet rebuilt the libvips package with that flag, so if you think there is a significant gain to be made from that vs. using an Image Magick library that pulls the first frame of the gif and processes that then let me know ( I've since written a GM/IM wrapper ( node gm's resizing method when it's just given a width just doesn't work, so I've had to `subClass` IM) to handle all GIF operations for now). The thing is, sharp just doesn't seem to be able to let go of memory for me unless it absolutely has to - it's super pronounced with GIF's because it will actually clear out memory to read it in, and you can see a drastic drop in used memory - but for a constant stream of smaller images, it just hovers at the danger zone and eventually runs OOM"}}