{"action":"created","comment":{"url":"https://api.github.com/repos/Shopify/sarama/pulls/comments/42071486","id":42071486,"diff_hunk":"@@ -320,7 +320,7 @@ func TestAsyncProducerEncoderFailures(t *testing.T) {\n \tleader.Returns(prodSuccess)\n \n \tconfig := NewConfig()\n-\tconfig.Producer.Flush.Messages = 3\n+\tconfig.Producer.Flush.Messages = 1","path":"async_producer_test.go","position":5,"original_position":5,"commit_id":"733c74f30dedcabeb062f1c01d115c790a7859ad","original_commit_id":"733c74f30dedcabeb062f1c01d115c790a7859ad","user":{"login":"wvanbergen","id":15870,"avatar_url":"https://avatars.githubusercontent.com/u/15870?v=3","gravatar_id":"","url":"https://api.github.com/users/wvanbergen","html_url":"https://github.com/wvanbergen","followers_url":"https://api.github.com/users/wvanbergen/followers","following_url":"https://api.github.com/users/wvanbergen/following{/other_user}","gists_url":"https://api.github.com/users/wvanbergen/gists{/gist_id}","starred_url":"https://api.github.com/users/wvanbergen/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wvanbergen/subscriptions","organizations_url":"https://api.github.com/users/wvanbergen/orgs","repos_url":"https://api.github.com/users/wvanbergen/repos","events_url":"https://api.github.com/users/wvanbergen/events{/privacy}","received_events_url":"https://api.github.com/users/wvanbergen/received_events","type":"User","site_admin":false},"body":"This was needed because the 2 failed to encode messages are not counted towards the batch anymore I assume?","created_at":"2015-10-15T00:25:41Z","updated_at":"2015-10-15T00:25:41Z","html_url":"https://github.com/Shopify/sarama/pull/550#discussion_r42071486","pull_request_url":"https://api.github.com/repos/Shopify/sarama/pulls/550","_links":{"self":{"href":"https://api.github.com/repos/Shopify/sarama/pulls/comments/42071486"},"html":{"href":"https://github.com/Shopify/sarama/pull/550#discussion_r42071486"},"pull_request":{"href":"https://api.github.com/repos/Shopify/sarama/pulls/550"}}},"pull_request":{"url":"https://api.github.com/repos/Shopify/sarama/pulls/550","id":46804740,"html_url":"https://github.com/Shopify/sarama/pull/550","diff_url":"https://github.com/Shopify/sarama/pull/550.diff","patch_url":"https://github.com/Shopify/sarama/pull/550.patch","issue_url":"https://api.github.com/repos/Shopify/sarama/issues/550","number":550,"state":"open","locked":false,"title":"Refactor the producer, part 2","user":{"login":"eapache","id":1706826,"avatar_url":"https://avatars.githubusercontent.com/u/1706826?v=3","gravatar_id":"","url":"https://api.github.com/users/eapache","html_url":"https://github.com/eapache","followers_url":"https://api.github.com/users/eapache/followers","following_url":"https://api.github.com/users/eapache/following{/other_user}","gists_url":"https://api.github.com/users/eapache/gists{/gist_id}","starred_url":"https://api.github.com/users/eapache/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/eapache/subscriptions","organizations_url":"https://api.github.com/users/eapache/orgs","repos_url":"https://api.github.com/users/eapache/repos","events_url":"https://api.github.com/users/eapache/events{/privacy}","received_events_url":"https://api.github.com/users/eapache/received_events","type":"User","site_admin":false},"body":"Second stand-alone chunk extracted from #544, (first chunk: #549). This uses the\r\n`produceSet` struct in the aggregator as well, and moves the `wouldOverflow` and\r\n`readyToFlush` methods to methods on the `produceSet`.\r\n\r\nKnock-on effects:\r\n - now that we do per-partition size tracking in the aggregator we can do much\r\n   more precise overflow checking (see the compressed-message-batch-size-limit\r\n   case in `wouldOverflow` which has changed) which will be more efficient in\r\n   high-volume scenarios\r\n - since the produceSet encodes immediately, messages which fail to encode are\r\n   now rejected from the aggregator and don't count towards batch size\r\n - we still have to iterate the messages in the flusher in order to reject those\r\n   which need retrying due to the state machine; for simplicity I add them to a\r\n   second produceSet still, which means all messages get encoded twice; this is\r\n   a definite major performance regression which will go away again in part 3\r\n   of this refactor\r\n\r\n@wvanbergen ","created_at":"2015-10-05T16:46:09Z","updated_at":"2015-10-15T00:25:41Z","closed_at":null,"merged_at":null,"merge_commit_sha":"c19b4805f8fb711bfe398b7038c4efb1d16c1702","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/Shopify/sarama/pulls/550/commits","review_comments_url":"https://api.github.com/repos/Shopify/sarama/pulls/550/comments","review_comment_url":"https://api.github.com/repos/Shopify/sarama/pulls/comments{/number}","comments_url":"https://api.github.com/repos/Shopify/sarama/issues/550/comments","statuses_url":"https://api.github.com/repos/Shopify/sarama/statuses/733c74f30dedcabeb062f1c01d115c790a7859ad","head":{"label":"Shopify:producer-refactor-2","ref":"producer-refactor-2","sha":"733c74f30dedcabeb062f1c01d115c790a7859ad","user":{"login":"Shopify","id":8085,"avatar_url":"https://avatars.githubusercontent.com/u/8085?v=3","gravatar_id":"","url":"https://api.github.com/users/Shopify","html_url":"https://github.com/Shopify","followers_url":"https://api.github.com/users/Shopify/followers","following_url":"https://api.github.com/users/Shopify/following{/other_user}","gists_url":"https://api.github.com/users/Shopify/gists{/gist_id}","starred_url":"https://api.github.com/users/Shopify/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Shopify/subscriptions","organizations_url":"https://api.github.com/users/Shopify/orgs","repos_url":"https://api.github.com/users/Shopify/repos","events_url":"https://api.github.com/users/Shopify/events{/privacy}","received_events_url":"https://api.github.com/users/Shopify/received_events","type":"Organization","site_admin":false},"repo":{"id":11206255,"name":"sarama","full_name":"Shopify/sarama","owner":{"login":"Shopify","id":8085,"avatar_url":"https://avatars.githubusercontent.com/u/8085?v=3","gravatar_id":"","url":"https://api.github.com/users/Shopify","html_url":"https://github.com/Shopify","followers_url":"https://api.github.com/users/Shopify/followers","following_url":"https://api.github.com/users/Shopify/following{/other_user}","gists_url":"https://api.github.com/users/Shopify/gists{/gist_id}","starred_url":"https://api.github.com/users/Shopify/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Shopify/subscriptions","organizations_url":"https://api.github.com/users/Shopify/orgs","repos_url":"https://api.github.com/users/Shopify/repos","events_url":"https://api.github.com/users/Shopify/events{/privacy}","received_events_url":"https://api.github.com/users/Shopify/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/Shopify/sarama","description":"Sarama is a Go library for Apache Kafka 0.8 ","fork":false,"url":"https://api.github.com/repos/Shopify/sarama","forks_url":"https://api.github.com/repos/Shopify/sarama/forks","keys_url":"https://api.github.com/repos/Shopify/sarama/keys{/key_id}","collaborators_url":"https://api.github.com/repos/Shopify/sarama/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/Shopify/sarama/teams","hooks_url":"https://api.github.com/repos/Shopify/sarama/hooks","issue_events_url":"https://api.github.com/repos/Shopify/sarama/issues/events{/number}","events_url":"https://api.github.com/repos/Shopify/sarama/events","assignees_url":"https://api.github.com/repos/Shopify/sarama/assignees{/user}","branches_url":"https://api.github.com/repos/Shopify/sarama/branches{/branch}","tags_url":"https://api.github.com/repos/Shopify/sarama/tags","blobs_url":"https://api.github.com/repos/Shopify/sarama/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/Shopify/sarama/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/Shopify/sarama/git/refs{/sha}","trees_url":"https://api.github.com/repos/Shopify/sarama/git/trees{/sha}","statuses_url":"https://api.github.com/repos/Shopify/sarama/statuses/{sha}","languages_url":"https://api.github.com/repos/Shopify/sarama/languages","stargazers_url":"https://api.github.com/repos/Shopify/sarama/stargazers","contributors_url":"https://api.github.com/repos/Shopify/sarama/contributors","subscribers_url":"https://api.github.com/repos/Shopify/sarama/subscribers","subscription_url":"https://api.github.com/repos/Shopify/sarama/subscription","commits_url":"https://api.github.com/repos/Shopify/sarama/commits{/sha}","git_commits_url":"https://api.github.com/repos/Shopify/sarama/git/commits{/sha}","comments_url":"https://api.github.com/repos/Shopify/sarama/comments{/number}","issue_comment_url":"https://api.github.com/repos/Shopify/sarama/issues/comments{/number}","contents_url":"https://api.github.com/repos/Shopify/sarama/contents/{+path}","compare_url":"https://api.github.com/repos/Shopify/sarama/compare/{base}...{head}","merges_url":"https://api.github.com/repos/Shopify/sarama/merges","archive_url":"https://api.github.com/repos/Shopify/sarama/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/Shopify/sarama/downloads","issues_url":"https://api.github.com/repos/Shopify/sarama/issues{/number}","pulls_url":"https://api.github.com/repos/Shopify/sarama/pulls{/number}","milestones_url":"https://api.github.com/repos/Shopify/sarama/milestones{/number}","notifications_url":"https://api.github.com/repos/Shopify/sarama/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/Shopify/sarama/labels{/name}","releases_url":"https://api.github.com/repos/Shopify/sarama/releases{/id}","created_at":"2013-07-05T18:52:38Z","updated_at":"2015-10-13T05:58:02Z","pushed_at":"2015-10-14T21:42:56Z","git_url":"git://github.com/Shopify/sarama.git","ssh_url":"git@github.com:Shopify/sarama.git","clone_url":"https://github.com/Shopify/sarama.git","svn_url":"https://github.com/Shopify/sarama","homepage":"http://shopify.github.io/sarama","size":6296,"stargazers_count":662,"watchers_count":662,"language":"Go","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":true,"forks_count":124,"mirror_url":null,"open_issues_count":21,"forks":124,"open_issues":21,"watchers":662,"default_branch":"master"}},"base":{"label":"Shopify:master","ref":"master","sha":"f0e1e8afb3b67d285dc988fc9c71ed26436827ac","user":{"login":"Shopify","id":8085,"avatar_url":"https://avatars.githubusercontent.com/u/8085?v=3","gravatar_id":"","url":"https://api.github.com/users/Shopify","html_url":"https://github.com/Shopify","followers_url":"https://api.github.com/users/Shopify/followers","following_url":"https://api.github.com/users/Shopify/following{/other_user}","gists_url":"https://api.github.com/users/Shopify/gists{/gist_id}","starred_url":"https://api.github.com/users/Shopify/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Shopify/subscriptions","organizations_url":"https://api.github.com/users/Shopify/orgs","repos_url":"https://api.github.com/users/Shopify/repos","events_url":"https://api.github.com/users/Shopify/events{/privacy}","received_events_url":"https://api.github.com/users/Shopify/received_events","type":"Organization","site_admin":false},"repo":{"id":11206255,"name":"sarama","full_name":"Shopify/sarama","owner":{"login":"Shopify","id":8085,"avatar_url":"https://avatars.githubusercontent.com/u/8085?v=3","gravatar_id":"","url":"https://api.github.com/users/Shopify","html_url":"https://github.com/Shopify","followers_url":"https://api.github.com/users/Shopify/followers","following_url":"https://api.github.com/users/Shopify/following{/other_user}","gists_url":"https://api.github.com/users/Shopify/gists{/gist_id}","starred_url":"https://api.github.com/users/Shopify/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Shopify/subscriptions","organizations_url":"https://api.github.com/users/Shopify/orgs","repos_url":"https://api.github.com/users/Shopify/repos","events_url":"https://api.github.com/users/Shopify/events{/privacy}","received_events_url":"https://api.github.com/users/Shopify/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/Shopify/sarama","description":"Sarama is a Go library for Apache Kafka 0.8 ","fork":false,"url":"https://api.github.com/repos/Shopify/sarama","forks_url":"https://api.github.com/repos/Shopify/sarama/forks","keys_url":"https://api.github.com/repos/Shopify/sarama/keys{/key_id}","collaborators_url":"https://api.github.com/repos/Shopify/sarama/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/Shopify/sarama/teams","hooks_url":"https://api.github.com/repos/Shopify/sarama/hooks","issue_events_url":"https://api.github.com/repos/Shopify/sarama/issues/events{/number}","events_url":"https://api.github.com/repos/Shopify/sarama/events","assignees_url":"https://api.github.com/repos/Shopify/sarama/assignees{/user}","branches_url":"https://api.github.com/repos/Shopify/sarama/branches{/branch}","tags_url":"https://api.github.com/repos/Shopify/sarama/tags","blobs_url":"https://api.github.com/repos/Shopify/sarama/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/Shopify/sarama/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/Shopify/sarama/git/refs{/sha}","trees_url":"https://api.github.com/repos/Shopify/sarama/git/trees{/sha}","statuses_url":"https://api.github.com/repos/Shopify/sarama/statuses/{sha}","languages_url":"https://api.github.com/repos/Shopify/sarama/languages","stargazers_url":"https://api.github.com/repos/Shopify/sarama/stargazers","contributors_url":"https://api.github.com/repos/Shopify/sarama/contributors","subscribers_url":"https://api.github.com/repos/Shopify/sarama/subscribers","subscription_url":"https://api.github.com/repos/Shopify/sarama/subscription","commits_url":"https://api.github.com/repos/Shopify/sarama/commits{/sha}","git_commits_url":"https://api.github.com/repos/Shopify/sarama/git/commits{/sha}","comments_url":"https://api.github.com/repos/Shopify/sarama/comments{/number}","issue_comment_url":"https://api.github.com/repos/Shopify/sarama/issues/comments{/number}","contents_url":"https://api.github.com/repos/Shopify/sarama/contents/{+path}","compare_url":"https://api.github.com/repos/Shopify/sarama/compare/{base}...{head}","merges_url":"https://api.github.com/repos/Shopify/sarama/merges","archive_url":"https://api.github.com/repos/Shopify/sarama/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/Shopify/sarama/downloads","issues_url":"https://api.github.com/repos/Shopify/sarama/issues{/number}","pulls_url":"https://api.github.com/repos/Shopify/sarama/pulls{/number}","milestones_url":"https://api.github.com/repos/Shopify/sarama/milestones{/number}","notifications_url":"https://api.github.com/repos/Shopify/sarama/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/Shopify/sarama/labels{/name}","releases_url":"https://api.github.com/repos/Shopify/sarama/releases{/id}","created_at":"2013-07-05T18:52:38Z","updated_at":"2015-10-13T05:58:02Z","pushed_at":"2015-10-14T21:42:56Z","git_url":"git://github.com/Shopify/sarama.git","ssh_url":"git@github.com:Shopify/sarama.git","clone_url":"https://github.com/Shopify/sarama.git","svn_url":"https://github.com/Shopify/sarama","homepage":"http://shopify.github.io/sarama","size":6296,"stargazers_count":662,"watchers_count":662,"language":"Go","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":true,"forks_count":124,"mirror_url":null,"open_issues_count":21,"forks":124,"open_issues":21,"watchers":662,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/Shopify/sarama/pulls/550"},"html":{"href":"https://github.com/Shopify/sarama/pull/550"},"issue":{"href":"https://api.github.com/repos/Shopify/sarama/issues/550"},"comments":{"href":"https://api.github.com/repos/Shopify/sarama/issues/550/comments"},"review_comments":{"href":"https://api.github.com/repos/Shopify/sarama/pulls/550/comments"},"review_comment":{"href":"https://api.github.com/repos/Shopify/sarama/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/Shopify/sarama/pulls/550/commits"},"statuses":{"href":"https://api.github.com/repos/Shopify/sarama/statuses/733c74f30dedcabeb062f1c01d115c790a7859ad"}}}}