{"action":"created","issue":{"url":"https://api.github.com/repos/celery/kombu/issues/462","labels_url":"https://api.github.com/repos/celery/kombu/issues/462/labels{/name}","comments_url":"https://api.github.com/repos/celery/kombu/issues/462/comments","events_url":"https://api.github.com/repos/celery/kombu/issues/462/events","html_url":"https://github.com/celery/kombu/issues/462","id":64230279,"number":462,"title":"redis transport poller descriptor leak","user":{"login":"fillest","id":103396,"avatar_url":"https://avatars.githubusercontent.com/u/103396?v=3","gravatar_id":"","url":"https://api.github.com/users/fillest","html_url":"https://github.com/fillest","followers_url":"https://api.github.com/users/fillest/followers","following_url":"https://api.github.com/users/fillest/following{/other_user}","gists_url":"https://api.github.com/users/fillest/gists{/gist_id}","starred_url":"https://api.github.com/users/fillest/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fillest/subscriptions","organizations_url":"https://api.github.com/users/fillest/orgs","repos_url":"https://api.github.com/users/fillest/repos","events_url":"https://api.github.com/users/fillest/events{/privacy}","received_events_url":"https://api.github.com/users/fillest/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2015-03-25T09:48:53Z","updated_at":"2015-04-17T15:32:21Z","closed_at":null,"body":"Hello. It seems there's descriptor leak in kombu/transport/redis.py. We use celery with redis, disk got full, redis crashed, celery lost its connection with redis and started to reconnect. After some time I found lots of the following error in the log:\r\n```\r\n[2015-03-25 09:41:48,438: WARNING/MainProcess] consumer: Connection to broker lost. Trying to re-establish the connection...\r\nTraceback (most recent call last):\r\n  File \"/home/ubuntu/venv/local/lib/python2.7/site-packages/celery/worker/consumer.py\", line 278, in start\r\n    blueprint.start(self)\r\n  File \"/home/ubuntu/venv/local/lib/python2.7/site-packages/celery/bootsteps.py\", line 123, in start\r\n    step.start(parent)\r\n  File \"/home/ubuntu/venv/local/lib/python2.7/site-packages/celery/worker/consumer.py\", line 478, in start\r\n    c.connection = c.connect()\r\n  File \"/home/ubuntu/venv/local/lib/python2.7/site-packages/celery/worker/consumer.py\", line 375, in connect\r\n    callback=maybe_shutdown,\r\n  File \"/home/ubuntu/venv/local/lib/python2.7/site-packages/kombu/connection.py\", line 367, in ensure_connection\r\n    retry_over_time(self.connect, self.recoverable_connection_errors,\r\n  File \"/home/ubuntu/venv/local/lib/python2.7/site-packages/kombu/utils/__init__.py\", line 322, in __get__\r\n    value = obj.__dict__[self.__name__] = self.__get(obj)\r\n  File \"/home/ubuntu/venv/local/lib/python2.7/site-packages/kombu/connection.py\", line 786, in recoverable_connection_errors\r\n    return self.transport.recoverable_connection_errors\r\n  File \"/home/ubuntu/venv/local/lib/python2.7/site-packages/kombu/connection.py\", line 769, in transport\r\n    self._transport = self.create_transport()\r\n  File \"/home/ubuntu/venv/local/lib/python2.7/site-packages/kombu/connection.py\", line 514, in create_transport\r\n    return self.get_transport_cls()(client=self)\r\n  File \"/home/ubuntu/venv/local/lib/python2.7/site-packages/kombu/transport/redis.py\", line 923, in __init__\r\n    self.cycle = MultiChannelPoller()\r\n  File \"/home/ubuntu/venv/local/lib/python2.7/site-packages/kombu/transport/redis.py\", line 237, in __init__\r\n    self.poller = poll()\r\n  File \"/home/ubuntu/venv/local/lib/python2.7/site-packages/kombu/utils/eventio.py\", line 265, in poll\r\n  File \"/home/ubuntu/venv/local/lib/python2.7/site-packages/kombu/utils/eventio.py\", line 74, in __init__\r\nIOError: [Errno 24] Too many open files\r\n```\r\nI've checked `lsof` and there's over a thousand of `anon_inode:[eventpoll]` of `celeryd` so I think it's a strong corellation.\r\nVersion is 3.0.24."},"comment":{"url":"https://api.github.com/repos/celery/kombu/issues/comments/94010054","html_url":"https://github.com/celery/kombu/issues/462#issuecomment-94010054","issue_url":"https://api.github.com/repos/celery/kombu/issues/462","id":94010054,"user":{"login":"fillest","id":103396,"avatar_url":"https://avatars.githubusercontent.com/u/103396?v=3","gravatar_id":"","url":"https://api.github.com/users/fillest","html_url":"https://github.com/fillest","followers_url":"https://api.github.com/users/fillest/followers","following_url":"https://api.github.com/users/fillest/following{/other_user}","gists_url":"https://api.github.com/users/fillest/gists{/gist_id}","starred_url":"https://api.github.com/users/fillest/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fillest/subscriptions","organizations_url":"https://api.github.com/users/fillest/orgs","repos_url":"https://api.github.com/users/fillest/repos","events_url":"https://api.github.com/users/fillest/events{/privacy}","received_events_url":"https://api.github.com/users/fillest/received_events","type":"User","site_admin":false},"created_at":"2015-04-17T15:32:21Z","updated_at":"2015-04-17T15:32:21Z","body":">So seems this is only related to reconnecting\r\nYes."}}