{"action":"created","issue":{"url":"https://api.github.com/repos/asmodehn/celeros/issues/5","labels_url":"https://api.github.com/repos/asmodehn/celeros/issues/5/labels{/name}","comments_url":"https://api.github.com/repos/asmodehn/celeros/issues/5/comments","events_url":"https://api.github.com/repos/asmodehn/celeros/issues/5/events","html_url":"https://github.com/asmodehn/celeros/issues/5","id":101350971,"number":5,"title":"Celery workers receive multiple tasks even though prefetch is set to 1","user":{"login":"heuristicus","id":636456,"avatar_url":"https://avatars.githubusercontent.com/u/636456?v=3","gravatar_id":"","url":"https://api.github.com/users/heuristicus","html_url":"https://github.com/heuristicus","followers_url":"https://api.github.com/users/heuristicus/followers","following_url":"https://api.github.com/users/heuristicus/following{/other_user}","gists_url":"https://api.github.com/users/heuristicus/gists{/gist_id}","starred_url":"https://api.github.com/users/heuristicus/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/heuristicus/subscriptions","organizations_url":"https://api.github.com/users/heuristicus/orgs","repos_url":"https://api.github.com/users/heuristicus/repos","events_url":"https://api.github.com/users/heuristicus/events{/privacy}","received_events_url":"https://api.github.com/users/heuristicus/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2015-08-17T06:44:32Z","updated_at":"2015-08-18T02:41:34Z","closed_at":null,"body":"When multiple tasks are sent to the robots, instead of ignoring any task after the first they receive, they instead prefetch another task, which we do not want.\r\n\r\nCurrent behaviour observed is that one worker (simulated robot) will accept a task, and then when another task is sent accept that as well. Once all workers have two tasks they no longer accept more. Once a task is completed, any tasks that were waiting in the queue are accepted."},"comment":{"url":"https://api.github.com/repos/asmodehn/celeros/issues/comments/132043148","html_url":"https://github.com/asmodehn/celeros/issues/5#issuecomment-132043148","issue_url":"https://api.github.com/repos/asmodehn/celeros/issues/5","id":132043148,"user":{"login":"heuristicus","id":636456,"avatar_url":"https://avatars.githubusercontent.com/u/636456?v=3","gravatar_id":"","url":"https://api.github.com/users/heuristicus","html_url":"https://github.com/heuristicus","followers_url":"https://api.github.com/users/heuristicus/followers","following_url":"https://api.github.com/users/heuristicus/following{/other_user}","gists_url":"https://api.github.com/users/heuristicus/gists{/gist_id}","starred_url":"https://api.github.com/users/heuristicus/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/heuristicus/subscriptions","organizations_url":"https://api.github.com/users/heuristicus/orgs","repos_url":"https://api.github.com/users/heuristicus/repos","events_url":"https://api.github.com/users/heuristicus/events{/privacy}","received_events_url":"https://api.github.com/users/heuristicus/received_events","type":"User","site_admin":false},"created_at":"2015-08-18T02:41:34Z","updated_at":"2015-08-18T02:41:34Z","body":"The qos object has a [`can_consume`](https://github.com/celery/kombu/blob/d8a7111627e5a1c937a8f8a6512beb14abb3e257/kombu/transport/virtual/__init__.py#L126) function which defines whether a message can be received from the incoming queue (or wherever it comes from).\r\n\r\nNot quite sure how the `prefetch_count` variable in the object works, but it looks like it is incremented and decremented when a message is received. If the prefetch count is zero, then the function will return true, because having a prefetch count of zero should result in the consumer consuming however many messages come in.\r\n\r\nAlternatively, the prefetch count can be nonzero, but the sizes of the delivered and dirty queues may result in the second part of the condition being fulfilled. It seems as though once a message is accepted by a worker, its hash is added to the dirty queue - if there is only a single message in the delivered queue and prefetch is 1, then we will also return true from the `can_consume` function.\r\n\r\nIn a way, this is expected behaviour, since one of the tasks has already been started, and we want to prefetch a single task. As such, we expect to have one task running and one task waiting to be run. In actuality, we want zero prefetched tasks, just a single task that starts running as soon as it is accepted (?)."}}