{"action":"created","issue":{"url":"https://api.github.com/repos/nesquena/backburner/issues/86","labels_url":"https://api.github.com/repos/nesquena/backburner/issues/86/labels{/name}","comments_url":"https://api.github.com/repos/nesquena/backburner/issues/86/comments","events_url":"https://api.github.com/repos/nesquena/backburner/issues/86/events","html_url":"https://github.com/nesquena/backburner/issues/86","id":60177956,"number":86,"title":"ThreadsOnFork multithreaded worker Timeout problem","user":{"login":"silentshade","id":886742,"avatar_url":"https://avatars.githubusercontent.com/u/886742?v=3","gravatar_id":"","url":"https://api.github.com/users/silentshade","html_url":"https://github.com/silentshade","followers_url":"https://api.github.com/users/silentshade/followers","following_url":"https://api.github.com/users/silentshade/following{/other_user}","gists_url":"https://api.github.com/users/silentshade/gists{/gist_id}","starred_url":"https://api.github.com/users/silentshade/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/silentshade/subscriptions","organizations_url":"https://api.github.com/users/silentshade/orgs","repos_url":"https://api.github.com/users/silentshade/repos","events_url":"https://api.github.com/users/silentshade/events{/privacy}","received_events_url":"https://api.github.com/users/silentshade/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":5,"created_at":"2015-03-07T00:13:33Z","updated_at":"2015-10-05T18:52:54Z","closed_at":null,"body":"We've ran into big problem using ```ThreadsOnFork``` with the job execution timeout. The ```Timeout::timeout``` is unsafe in multithreaded environment.\r\nGood explanation here http://blog.headius.com/2008/02/ruby-threadraise-threadkill-timeoutrb.html\r\nSidekiq similar issue, they removed timeout support completely https://github.com/mperham/sidekiq/issues/862#event-245331424\r\nI tried to locate the issue myself.\r\n\r\n**Environment**\r\nI've made a job class with timeout of 30 seconds, and code that runs from 1 to 3 minutes. The code includes some db access, logging, throwing and rescuing exceptions, setting and getting thread variables, etc..\r\nThe tube workers (threads) are  limited to 30.\r\n\r\n**On ruby 1.9.3**\r\nUntested yet, but i'm thinking it will be same as 2.0.0\r\n\r\n**On ruby 2.0.0**\r\nLikely because the timeout hits some rescue block somewhere inside the code, threads(workers) die very fast. In about one minute workers count decreased from 30 to 1.\r\nAs a i get following warnings in the console ```log writing failed. execution expired```, i suppose that timeout hits some rescue block within system ```Logger#write```, which is used heavily in code. Its another bug with this method, explained here: https://bugs.ruby-lang.org/issues/9115\r\nComments there are kind of unclear, but seems that in ruby 2.1 Timeout::Error is not rescued by default unless explicitly given as rescue argument.\r\n\r\n**On ruby 2.1.3**\r\n```Logger#write``` is not a problem anymore, threads are living much longer. However they die anyway after some time, timeout is hitting them somewhere in rescue block. Within half an hour workers count decreased from 30 to 1..\r\n\r\nWe need to consider another flow to timeout code execution and clean up within threaded worker. Any thoughts?\r\nInviting @nesquena :)\r\n"},"comment":{"url":"https://api.github.com/repos/nesquena/backburner/issues/comments/145630702","html_url":"https://github.com/nesquena/backburner/issues/86#issuecomment-145630702","issue_url":"https://api.github.com/repos/nesquena/backburner/issues/86","id":145630702,"user":{"login":"contentfree","id":10958,"avatar_url":"https://avatars.githubusercontent.com/u/10958?v=3","gravatar_id":"","url":"https://api.github.com/users/contentfree","html_url":"https://github.com/contentfree","followers_url":"https://api.github.com/users/contentfree/followers","following_url":"https://api.github.com/users/contentfree/following{/other_user}","gists_url":"https://api.github.com/users/contentfree/gists{/gist_id}","starred_url":"https://api.github.com/users/contentfree/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/contentfree/subscriptions","organizations_url":"https://api.github.com/users/contentfree/orgs","repos_url":"https://api.github.com/users/contentfree/repos","events_url":"https://api.github.com/users/contentfree/events{/privacy}","received_events_url":"https://api.github.com/users/contentfree/received_events","type":"User","site_admin":false},"created_at":"2015-10-05T18:52:54Z","updated_at":"2015-10-05T18:52:54Z","body":"@nesquena:  In addition to a) continue to handle a timeout of 0 and b) reverting to `ttr - 1`, should we also change `resolve_respond_timeout` to add one to the `ttr` when enqueuing as well so that we don't have to worry about things like a `ttr` of 1 leading to no timeout (since subtracting one would result in jobs never timing out?  \r\n\r\nIt get messy… I'm liking the removal of the Timeout usage altogether and leaving it up to the job to do timeouts. However, it brings up its own set of difficulties (such as retrying with hooks and delay upon job failure / timeout; async proxies; automagic async)...\r\n\r\n"}}