{"action":"created","issue":{"url":"https://api.github.com/repos/mumrah/kafka-python/issues/331","labels_url":"https://api.github.com/repos/mumrah/kafka-python/issues/331/labels{/name}","comments_url":"https://api.github.com/repos/mumrah/kafka-python/issues/331/comments","events_url":"https://api.github.com/repos/mumrah/kafka-python/issues/331/events","html_url":"https://github.com/mumrah/kafka-python/pull/331","id":58901615,"number":331,"title":"Async producer retries for failed messages","user":{"login":"vshlapakov","id":448111,"avatar_url":"https://avatars.githubusercontent.com/u/448111?v=3","gravatar_id":"","url":"https://api.github.com/users/vshlapakov","html_url":"https://github.com/vshlapakov","followers_url":"https://api.github.com/users/vshlapakov/followers","following_url":"https://api.github.com/users/vshlapakov/following{/other_user}","gists_url":"https://api.github.com/users/vshlapakov/gists{/gist_id}","starred_url":"https://api.github.com/users/vshlapakov/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vshlapakov/subscriptions","organizations_url":"https://api.github.com/users/vshlapakov/orgs","repos_url":"https://api.github.com/users/vshlapakov/repos","events_url":"https://api.github.com/users/vshlapakov/events{/privacy}","received_events_url":"https://api.github.com/users/vshlapakov/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/mumrah/kafka-python/labels/producer","name":"producer","color":"fef2c0"}],"state":"closed","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/mumrah/kafka-python/milestones/5","html_url":"https://github.com/mumrah/kafka-python/milestones/0.9.4%20Release","labels_url":"https://api.github.com/repos/mumrah/kafka-python/milestones/5/labels","id":1035587,"number":5,"title":"0.9.4 Release","description":"Issues / PRs to be included in 0.9.4 release","creator":{"login":"dpkp","id":843444,"avatar_url":"https://avatars.githubusercontent.com/u/843444?v=3","gravatar_id":"","url":"https://api.github.com/users/dpkp","html_url":"https://github.com/dpkp","followers_url":"https://api.github.com/users/dpkp/followers","following_url":"https://api.github.com/users/dpkp/following{/other_user}","gists_url":"https://api.github.com/users/dpkp/gists{/gist_id}","starred_url":"https://api.github.com/users/dpkp/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dpkp/subscriptions","organizations_url":"https://api.github.com/users/dpkp/orgs","repos_url":"https://api.github.com/users/dpkp/repos","events_url":"https://api.github.com/users/dpkp/events{/privacy}","received_events_url":"https://api.github.com/users/dpkp/received_events","type":"User","site_admin":false},"open_issues":11,"closed_issues":26,"state":"open","created_at":"2015-03-24T02:51:18Z","updated_at":"2015-06-07T18:49:01Z","due_on":null,"closed_at":null},"comments":23,"created_at":"2015-02-25T13:31:10Z","updated_at":"2015-06-07T20:05:36Z","closed_at":"2015-06-04T20:21:12Z","pull_request":{"url":"https://api.github.com/repos/mumrah/kafka-python/pulls/331","html_url":"https://github.com/mumrah/kafka-python/pull/331","diff_url":"https://github.com/mumrah/kafka-python/pull/331.diff","patch_url":"https://github.com/mumrah/kafka-python/pull/331.patch"},"body":"Another approach to add retries for failed messages (for asynchronous mode only).\r\n\r\n1. I've extended `ProduceRequest` a little bit by adding complete retries count to the fields.\r\nIt's backward compatible, you can miss this param when creating initial ProduceRequest, it's 0 initially.\r\n(It's not very clean solution in terms of code, but storing retries count per request sounds reasonable.)\r\n\r\n2. I've added several retry params for producers also:\r\n  * `batch_retry_backoff_ms` - backoff timeout to wait before next retry (300 by default);\r\n  * `batch_retries_limit` - upper limit of retries per request \r\n  (`5` by default, `None` means infinite retries, `0` means no retries).\r\n\r\n3. I've used `FailedPayloadsException` from #55 to get failed requests and increment retries for it, or drop it depending on settings. \r\n\r\nWhat do you think about the approach? I'm going to add some additional tests for it asap."},"comment":{"url":"https://api.github.com/repos/mumrah/kafka-python/issues/comments/109794352","html_url":"https://github.com/mumrah/kafka-python/pull/331#issuecomment-109794352","issue_url":"https://api.github.com/repos/mumrah/kafka-python/issues/331","id":109794352,"user":{"login":"vshlapakov","id":448111,"avatar_url":"https://avatars.githubusercontent.com/u/448111?v=3","gravatar_id":"","url":"https://api.github.com/users/vshlapakov","html_url":"https://github.com/vshlapakov","followers_url":"https://api.github.com/users/vshlapakov/followers","following_url":"https://api.github.com/users/vshlapakov/following{/other_user}","gists_url":"https://api.github.com/users/vshlapakov/gists{/gist_id}","starred_url":"https://api.github.com/users/vshlapakov/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vshlapakov/subscriptions","organizations_url":"https://api.github.com/users/vshlapakov/orgs","repos_url":"https://api.github.com/users/vshlapakov/repos","events_url":"https://api.github.com/users/vshlapakov/events{/privacy}","received_events_url":"https://api.github.com/users/vshlapakov/received_events","type":"User","site_admin":false},"created_at":"2015-06-07T20:05:36Z","updated_at":"2015-06-07T20:05:36Z","body":"Cool, thank you @dpkp! If I have some additional time, I'll fix other issues (like additional tests and docstrings) as well in a separate PR."}}