{"action":"created","issue":{"url":"https://api.github.com/repos/mperham/sidekiq/issues/2107","labels_url":"https://api.github.com/repos/mperham/sidekiq/issues/2107/labels{/name}","comments_url":"https://api.github.com/repos/mperham/sidekiq/issues/2107/comments","events_url":"https://api.github.com/repos/mperham/sidekiq/issues/2107/events","html_url":"https://github.com/mperham/sidekiq/issues/2107","id":53088078,"number":2107,"title":"Memory leak if worker raises Exception instead of StandardError (or subclasses)","user":{"login":"deborasetton","id":868639,"avatar_url":"https://avatars.githubusercontent.com/u/868639?v=3","gravatar_id":"","url":"https://api.github.com/users/deborasetton","html_url":"https://github.com/deborasetton","followers_url":"https://api.github.com/users/deborasetton/followers","following_url":"https://api.github.com/users/deborasetton/following{/other_user}","gists_url":"https://api.github.com/users/deborasetton/gists{/gist_id}","starred_url":"https://api.github.com/users/deborasetton/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/deborasetton/subscriptions","organizations_url":"https://api.github.com/users/deborasetton/orgs","repos_url":"https://api.github.com/users/deborasetton/repos","events_url":"https://api.github.com/users/deborasetton/events{/privacy}","received_events_url":"https://api.github.com/users/deborasetton/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":7,"created_at":"2014-12-30T02:17:09Z","updated_at":"2015-01-05T13:31:16Z","closed_at":"2015-01-04T01:16:51Z","body":"Hi,\r\n\r\nSorry if the description below is too long!, but the **TL;DR** is basically the title and what follows is my adventure in Debugging Land.\r\n\r\nI'm opening this issue to **a)** ask if this is a known problem and **b)** understand the underlying reason so it won't bite me in the future.\r\n<hr>\r\n\r\nI've been tracking down a memory leak in a Sidekiq process. I started debugging my original Rails application, then reduced it to a smaller, minimal Rails app, then to an even simpler Sinatra app and, finally, thanks to the example by @grosser in #1353, to just one file and a few lines of code:\r\n\r\nhttps://gist.github.com/deborasetton/713229c641ba13c6b1b0#file-leak-rb\r\n\r\nThe results:\r\n- When running this script with 5000 jobs and LEAK=0, the memory usage peaks at **~34MB**;\r\n- When running the same 5000 jobs with LEAK=1, the memory usage goes beyond **250MB**!\r\n\r\nAfter some time debugging both Sidekiq and Celluloid to find the difference in the code path for these two cases, it looks like [this line](https://github.com/celluloid/celluloid/blob/master/lib/celluloid/actor.rb#L167) from Celluloid, if removed, would solve the problem:\r\n\r\n```ruby\r\n# celluloid/actor.rb:167\r\nraise unless ex.is_a? StandardError\r\n```\r\n\r\nWhen I remove it and re-run the test script, it seems to work, but I can't explain **why** this fixes my problem. I know very little about Sidekiq and Celluloid internals, so I couldn't find out where this `raise` gets `rescued` (even when stepping into the code with byebug/debugger), or even if it happens within Sidekiq or Celluloid, because, at this point, it seems the code keeps bouncing between the two and a lot of Actors and stuff... So I'm kind of stuck at my investigation.\r\n\r\nHas anyone ever come accross this issue before? If not, I would still appreciate any pointers on where to look next!\r\n\r\n<hr>\r\n\r\nExtra information:\r\n\r\n- I've now learned the hard way that I should be raising something else instead of `Exception`, but I'm still curious to understand why this has such different consequences in this case.\r\n\r\n- I've tested this code with Ruby 2.0.0, 2.1.4 and 2.2.0, and with Sidekiq 3.2.6 and 3.3.0, all with similar results.\r\n\r\n- If the aforementioned Celluloid line is removed, Celluloid specs still pass, so I wasn't able to find a spec or explanation for why it exists, but it's been there since commit [2394d7](https://github.com/celluloid/celluloid/commit/2394d7).\r\n\r\n- I got to this line in Celluloid after following @krasnoukhov's [blog post on profiling Sidekiq](http://blog.krasnoukhov.com/posts/2013/01/07/profiling-memory-leaky-sidekiq-applications-with-ruby-2.1.html), so props to it!\r\n\r\n- The leaked memory is not released after the jobs are done, but it seems this is normal behavior.\r\n\r\nIf you got here, thanks so much for your patience! Can you help me? :)\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/mperham/sidekiq/issues/comments/68707518","html_url":"https://github.com/mperham/sidekiq/issues/2107#issuecomment-68707518","issue_url":"https://api.github.com/repos/mperham/sidekiq/issues/2107","id":68707518,"user":{"login":"deborasetton","id":868639,"avatar_url":"https://avatars.githubusercontent.com/u/868639?v=3","gravatar_id":"","url":"https://api.github.com/users/deborasetton","html_url":"https://github.com/deborasetton","followers_url":"https://api.github.com/users/deborasetton/followers","following_url":"https://api.github.com/users/deborasetton/following{/other_user}","gists_url":"https://api.github.com/users/deborasetton/gists{/gist_id}","starred_url":"https://api.github.com/users/deborasetton/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/deborasetton/subscriptions","organizations_url":"https://api.github.com/users/deborasetton/orgs","repos_url":"https://api.github.com/users/deborasetton/repos","events_url":"https://api.github.com/users/deborasetton/events{/privacy}","received_events_url":"https://api.github.com/users/deborasetton/received_events","type":"User","site_admin":false},"created_at":"2015-01-05T13:31:16Z","updated_at":"2015-01-05T13:31:16Z","body":"Thanks, @sb8244 and @tarcieri! That was really quick and the discussion that lead to the fix was very informative!"}}