{"action":"created","issue":{"url":"https://api.github.com/repos/celery/celery/issues/2429","labels_url":"https://api.github.com/repos/celery/celery/issues/2429/labels{/name}","comments_url":"https://api.github.com/repos/celery/celery/issues/2429/comments","events_url":"https://api.github.com/repos/celery/celery/issues/2429/events","html_url":"https://github.com/celery/celery/issues/2429","id":52549073,"number":2429,"title":"Celery Hanging with FUTEX_WAIT_PRIVATE","user":{"login":"maxsmythe","id":5998310,"avatar_url":"https://avatars.githubusercontent.com/u/5998310?v=3","gravatar_id":"","url":"https://api.github.com/users/maxsmythe","html_url":"https://github.com/maxsmythe","followers_url":"https://api.github.com/users/maxsmythe/followers","following_url":"https://api.github.com/users/maxsmythe/following{/other_user}","gists_url":"https://api.github.com/users/maxsmythe/gists{/gist_id}","starred_url":"https://api.github.com/users/maxsmythe/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/maxsmythe/subscriptions","organizations_url":"https://api.github.com/users/maxsmythe/orgs","repos_url":"https://api.github.com/users/maxsmythe/repos","events_url":"https://api.github.com/users/maxsmythe/events{/privacy}","received_events_url":"https://api.github.com/users/maxsmythe/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/celery/celery/labels/Issue%20Type:%20Bug","name":"Issue Type: Bug","color":"e11d21"},{"url":"https://api.github.com/repos/celery/celery/labels/Worker%20Hangs","name":"Worker Hangs","color":"E60026"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2014-12-20T02:41:54Z","updated_at":"2015-07-13T22:07:16Z","closed_at":null,"body":"We are encountering a problem running Celery 3.1.9\r\n\r\nAfter running for a seemingly random amount of time, Celery will hang.  There is nothing obviously wrong in the logs, the worker will continue to queue jobs locally, but they will not be processed.\r\n\r\nAfter running strace, we found that all worker processes were hanging with the same message:\r\n\r\n```\r\nsudo strace -p 25389\r\nProcess 25389 attached - interrupt to quit\r\nfutex(0x44fb218, FUTEX_WAIT_PRIVATE, 2, NULL\r\nProcess 25389 detached\r\n```\r\n\r\nThis looks similar to https://github.com/celery/celery/issues/572\r\n\r\nWe were going to try enabling CELERYD_FORCE_EXECV, but that looks to be contraindicated:\r\nhttp://celery.readthedocs.org/en/latest/changelog.html\r\n\r\nAfter further research, it looks like you have addressed this bug by using async IO for consumer communication as of 3.1.0.  We do appear to be using the new async prefork pool: RabbitMQ is our broker and Worker.use_eventloop is True.\r\n\r\nDo you know why we might still be experiencing deadlocks even with the new event loop?  Here are the options with which we are launching celery workers:\r\n\r\ncelery -A app worker -l info -E -Q default -Ofair --concurrency=5 --hostname=%n.ourworker.com --maxtasksperchild=50\r\n"},"comment":{"url":"https://api.github.com/repos/celery/celery/issues/comments/121076242","html_url":"https://github.com/celery/celery/issues/2429#issuecomment-121076242","issue_url":"https://api.github.com/repos/celery/celery/issues/2429","id":121076242,"user":{"login":"domenkozar","id":126339,"avatar_url":"https://avatars.githubusercontent.com/u/126339?v=3","gravatar_id":"","url":"https://api.github.com/users/domenkozar","html_url":"https://github.com/domenkozar","followers_url":"https://api.github.com/users/domenkozar/followers","following_url":"https://api.github.com/users/domenkozar/following{/other_user}","gists_url":"https://api.github.com/users/domenkozar/gists{/gist_id}","starred_url":"https://api.github.com/users/domenkozar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/domenkozar/subscriptions","organizations_url":"https://api.github.com/users/domenkozar/orgs","repos_url":"https://api.github.com/users/domenkozar/repos","events_url":"https://api.github.com/users/domenkozar/events{/privacy}","received_events_url":"https://api.github.com/users/domenkozar/received_events","type":"User","site_admin":false},"created_at":"2015-07-13T22:07:16Z","updated_at":"2015-07-13T22:07:16Z","body":"@derega did you ever figure out the cause? We have very similar stacktraces that hang."}}