{"action":"created","comment":{"url":"https://api.github.com/repos/Shopify/camus/pulls/comments/30287110","id":30287110,"diff_hunk":"@@ -40,15 +45,28 @@ private void submitCountersToStatsd(Job job) throws IOException {\n         }\n     }\n \n-    public static Boolean getStatsdEnabled(Job job) {\n-        return job.getConfiguration().getBoolean(STATSD_ENABLED, false);\n+    public static void logEventDelay(JobContext context, String topic, CamusWrapper wrap) {\n+        if (getStatsdEnabled(context.getConfiguration())) {\n+            StatsDClient statsd = new NonBlockingStatsDClient(","path":"camus-etl-kafka/src/main/java/com/linkedin/camus/etl/kafka/reporter/StatsdReporter.java","position":40,"original_position":40,"commit_id":"ce0e853c893321e86425e57fa5b0b57addd7ba04","original_commit_id":"ce0e853c893321e86425e57fa5b0b57addd7ba04","user":{"login":"drdee","id":109400,"avatar_url":"https://avatars.githubusercontent.com/u/109400?v=3","gravatar_id":"","url":"https://api.github.com/users/drdee","html_url":"https://github.com/drdee","followers_url":"https://api.github.com/users/drdee/followers","following_url":"https://api.github.com/users/drdee/following{/other_user}","gists_url":"https://api.github.com/users/drdee/gists{/gist_id}","starred_url":"https://api.github.com/users/drdee/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/drdee/subscriptions","organizations_url":"https://api.github.com/users/drdee/orgs","repos_url":"https://api.github.com/users/drdee/repos","events_url":"https://api.github.com/users/drdee/events{/privacy}","received_events_url":"https://api.github.com/users/drdee/received_events","type":"User","site_admin":false},"body":"You are instantiating `statsd` for every single record it sees and then you only use it if the delay is longer than a hour.  Why not instantiate `statsd` once inside `EtlRecordReader`? That might mean a small refactor but I think this is very expensive. If the refactor is not feasible then  at the very least, you should only instantiate `statsd` if the delay is more than a hour.","created_at":"2015-05-13T23:44:34Z","updated_at":"2015-05-13T23:44:34Z","html_url":"https://github.com/Shopify/camus/pull/38#discussion_r30287110","pull_request_url":"https://api.github.com/repos/Shopify/camus/pulls/38","_links":{"self":{"href":"https://api.github.com/repos/Shopify/camus/pulls/comments/30287110"},"html":{"href":"https://github.com/Shopify/camus/pull/38#discussion_r30287110"},"pull_request":{"href":"https://api.github.com/repos/Shopify/camus/pulls/38"}}},"pull_request":{"url":"https://api.github.com/repos/Shopify/camus/pulls/38","id":35371414,"html_url":"https://github.com/Shopify/camus/pull/38","diff_url":"https://github.com/Shopify/camus/pull/38.diff","patch_url":"https://github.com/Shopify/camus/pull/38.patch","issue_url":"https://api.github.com/repos/Shopify/camus/issues/38","number":38,"state":"open","locked":false,"title":"Log delay from camus","user":{"login":"yagnik","id":777296,"avatar_url":"https://avatars.githubusercontent.com/u/777296?v=3","gravatar_id":"","url":"https://api.github.com/users/yagnik","html_url":"https://github.com/yagnik","followers_url":"https://api.github.com/users/yagnik/followers","following_url":"https://api.github.com/users/yagnik/following{/other_user}","gists_url":"https://api.github.com/users/yagnik/gists{/gist_id}","starred_url":"https://api.github.com/users/yagnik/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/yagnik/subscriptions","organizations_url":"https://api.github.com/users/yagnik/orgs","repos_url":"https://api.github.com/users/yagnik/repos","events_url":"https://api.github.com/users/yagnik/events{/privacy}","received_events_url":"https://api.github.com/users/yagnik/received_events","type":"User","site_admin":false},"body":"This will log delay to :dog: if it is over 1 hour from load time. It will allow us to see which topics are regularly late and we can then dig into why.\r\n\r\nI'm already re-writing reporting module for camus for confluent so I don't plan to move this upstream to linkedin. \r\n\r\n@drdee @kmtaylor-github \r\ncc @Shopify/data-engineers ","created_at":"2015-05-13T21:13:43Z","updated_at":"2015-05-13T23:44:34Z","closed_at":null,"merged_at":null,"merge_commit_sha":"3cf08fe2f483ad9031b695dd28bdde96407ace69","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/Shopify/camus/pulls/38/commits","review_comments_url":"https://api.github.com/repos/Shopify/camus/pulls/38/comments","review_comment_url":"https://api.github.com/repos/Shopify/camus/pulls/comments{/number}","comments_url":"https://api.github.com/repos/Shopify/camus/issues/38/comments","statuses_url":"https://api.github.com/repos/Shopify/camus/statuses/ce0e853c893321e86425e57fa5b0b57addd7ba04","head":{"label":"Shopify:log-delay","ref":"log-delay","sha":"ce0e853c893321e86425e57fa5b0b57addd7ba04","user":{"login":"Shopify","id":8085,"avatar_url":"https://avatars.githubusercontent.com/u/8085?v=3","gravatar_id":"","url":"https://api.github.com/users/Shopify","html_url":"https://github.com/Shopify","followers_url":"https://api.github.com/users/Shopify/followers","following_url":"https://api.github.com/users/Shopify/following{/other_user}","gists_url":"https://api.github.com/users/Shopify/gists{/gist_id}","starred_url":"https://api.github.com/users/Shopify/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Shopify/subscriptions","organizations_url":"https://api.github.com/users/Shopify/orgs","repos_url":"https://api.github.com/users/Shopify/repos","events_url":"https://api.github.com/users/Shopify/events{/privacy}","received_events_url":"https://api.github.com/users/Shopify/received_events","type":"Organization","site_admin":false},"repo":{"id":20409468,"name":"camus","full_name":"Shopify/camus","owner":{"login":"Shopify","id":8085,"avatar_url":"https://avatars.githubusercontent.com/u/8085?v=3","gravatar_id":"","url":"https://api.github.com/users/Shopify","html_url":"https://github.com/Shopify","followers_url":"https://api.github.com/users/Shopify/followers","following_url":"https://api.github.com/users/Shopify/following{/other_user}","gists_url":"https://api.github.com/users/Shopify/gists{/gist_id}","starred_url":"https://api.github.com/users/Shopify/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Shopify/subscriptions","organizations_url":"https://api.github.com/users/Shopify/orgs","repos_url":"https://api.github.com/users/Shopify/repos","events_url":"https://api.github.com/users/Shopify/events{/privacy}","received_events_url":"https://api.github.com/users/Shopify/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/Shopify/camus","description":null,"fork":true,"url":"https://api.github.com/repos/Shopify/camus","forks_url":"https://api.github.com/repos/Shopify/camus/forks","keys_url":"https://api.github.com/repos/Shopify/camus/keys{/key_id}","collaborators_url":"https://api.github.com/repos/Shopify/camus/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/Shopify/camus/teams","hooks_url":"https://api.github.com/repos/Shopify/camus/hooks","issue_events_url":"https://api.github.com/repos/Shopify/camus/issues/events{/number}","events_url":"https://api.github.com/repos/Shopify/camus/events","assignees_url":"https://api.github.com/repos/Shopify/camus/assignees{/user}","branches_url":"https://api.github.com/repos/Shopify/camus/branches{/branch}","tags_url":"https://api.github.com/repos/Shopify/camus/tags","blobs_url":"https://api.github.com/repos/Shopify/camus/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/Shopify/camus/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/Shopify/camus/git/refs{/sha}","trees_url":"https://api.github.com/repos/Shopify/camus/git/trees{/sha}","statuses_url":"https://api.github.com/repos/Shopify/camus/statuses/{sha}","languages_url":"https://api.github.com/repos/Shopify/camus/languages","stargazers_url":"https://api.github.com/repos/Shopify/camus/stargazers","contributors_url":"https://api.github.com/repos/Shopify/camus/contributors","subscribers_url":"https://api.github.com/repos/Shopify/camus/subscribers","subscription_url":"https://api.github.com/repos/Shopify/camus/subscription","commits_url":"https://api.github.com/repos/Shopify/camus/commits{/sha}","git_commits_url":"https://api.github.com/repos/Shopify/camus/git/commits{/sha}","comments_url":"https://api.github.com/repos/Shopify/camus/comments{/number}","issue_comment_url":"https://api.github.com/repos/Shopify/camus/issues/comments{/number}","contents_url":"https://api.github.com/repos/Shopify/camus/contents/{+path}","compare_url":"https://api.github.com/repos/Shopify/camus/compare/{base}...{head}","merges_url":"https://api.github.com/repos/Shopify/camus/merges","archive_url":"https://api.github.com/repos/Shopify/camus/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/Shopify/camus/downloads","issues_url":"https://api.github.com/repos/Shopify/camus/issues{/number}","pulls_url":"https://api.github.com/repos/Shopify/camus/pulls{/number}","milestones_url":"https://api.github.com/repos/Shopify/camus/milestones{/number}","notifications_url":"https://api.github.com/repos/Shopify/camus/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/Shopify/camus/labels{/name}","releases_url":"https://api.github.com/repos/Shopify/camus/releases{/id}","created_at":"2014-06-02T15:42:19Z","updated_at":"2015-04-28T15:35:27Z","pushed_at":"2015-05-13T21:24:21Z","git_url":"git://github.com/Shopify/camus.git","ssh_url":"git@github.com:Shopify/camus.git","clone_url":"https://github.com/Shopify/camus.git","svn_url":"https://github.com/Shopify/camus","homepage":null,"size":42257,"stargazers_count":1,"watchers_count":1,"language":"Java","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":3,"forks":0,"open_issues":3,"watchers":1,"default_branch":"master"}},"base":{"label":"Shopify:master","ref":"master","sha":"0ace8035b8d0cac79faf0bad7ba36c5792d2734f","user":{"login":"Shopify","id":8085,"avatar_url":"https://avatars.githubusercontent.com/u/8085?v=3","gravatar_id":"","url":"https://api.github.com/users/Shopify","html_url":"https://github.com/Shopify","followers_url":"https://api.github.com/users/Shopify/followers","following_url":"https://api.github.com/users/Shopify/following{/other_user}","gists_url":"https://api.github.com/users/Shopify/gists{/gist_id}","starred_url":"https://api.github.com/users/Shopify/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Shopify/subscriptions","organizations_url":"https://api.github.com/users/Shopify/orgs","repos_url":"https://api.github.com/users/Shopify/repos","events_url":"https://api.github.com/users/Shopify/events{/privacy}","received_events_url":"https://api.github.com/users/Shopify/received_events","type":"Organization","site_admin":false},"repo":{"id":20409468,"name":"camus","full_name":"Shopify/camus","owner":{"login":"Shopify","id":8085,"avatar_url":"https://avatars.githubusercontent.com/u/8085?v=3","gravatar_id":"","url":"https://api.github.com/users/Shopify","html_url":"https://github.com/Shopify","followers_url":"https://api.github.com/users/Shopify/followers","following_url":"https://api.github.com/users/Shopify/following{/other_user}","gists_url":"https://api.github.com/users/Shopify/gists{/gist_id}","starred_url":"https://api.github.com/users/Shopify/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Shopify/subscriptions","organizations_url":"https://api.github.com/users/Shopify/orgs","repos_url":"https://api.github.com/users/Shopify/repos","events_url":"https://api.github.com/users/Shopify/events{/privacy}","received_events_url":"https://api.github.com/users/Shopify/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/Shopify/camus","description":null,"fork":true,"url":"https://api.github.com/repos/Shopify/camus","forks_url":"https://api.github.com/repos/Shopify/camus/forks","keys_url":"https://api.github.com/repos/Shopify/camus/keys{/key_id}","collaborators_url":"https://api.github.com/repos/Shopify/camus/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/Shopify/camus/teams","hooks_url":"https://api.github.com/repos/Shopify/camus/hooks","issue_events_url":"https://api.github.com/repos/Shopify/camus/issues/events{/number}","events_url":"https://api.github.com/repos/Shopify/camus/events","assignees_url":"https://api.github.com/repos/Shopify/camus/assignees{/user}","branches_url":"https://api.github.com/repos/Shopify/camus/branches{/branch}","tags_url":"https://api.github.com/repos/Shopify/camus/tags","blobs_url":"https://api.github.com/repos/Shopify/camus/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/Shopify/camus/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/Shopify/camus/git/refs{/sha}","trees_url":"https://api.github.com/repos/Shopify/camus/git/trees{/sha}","statuses_url":"https://api.github.com/repos/Shopify/camus/statuses/{sha}","languages_url":"https://api.github.com/repos/Shopify/camus/languages","stargazers_url":"https://api.github.com/repos/Shopify/camus/stargazers","contributors_url":"https://api.github.com/repos/Shopify/camus/contributors","subscribers_url":"https://api.github.com/repos/Shopify/camus/subscribers","subscription_url":"https://api.github.com/repos/Shopify/camus/subscription","commits_url":"https://api.github.com/repos/Shopify/camus/commits{/sha}","git_commits_url":"https://api.github.com/repos/Shopify/camus/git/commits{/sha}","comments_url":"https://api.github.com/repos/Shopify/camus/comments{/number}","issue_comment_url":"https://api.github.com/repos/Shopify/camus/issues/comments{/number}","contents_url":"https://api.github.com/repos/Shopify/camus/contents/{+path}","compare_url":"https://api.github.com/repos/Shopify/camus/compare/{base}...{head}","merges_url":"https://api.github.com/repos/Shopify/camus/merges","archive_url":"https://api.github.com/repos/Shopify/camus/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/Shopify/camus/downloads","issues_url":"https://api.github.com/repos/Shopify/camus/issues{/number}","pulls_url":"https://api.github.com/repos/Shopify/camus/pulls{/number}","milestones_url":"https://api.github.com/repos/Shopify/camus/milestones{/number}","notifications_url":"https://api.github.com/repos/Shopify/camus/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/Shopify/camus/labels{/name}","releases_url":"https://api.github.com/repos/Shopify/camus/releases{/id}","created_at":"2014-06-02T15:42:19Z","updated_at":"2015-04-28T15:35:27Z","pushed_at":"2015-05-13T21:24:21Z","git_url":"git://github.com/Shopify/camus.git","ssh_url":"git@github.com:Shopify/camus.git","clone_url":"https://github.com/Shopify/camus.git","svn_url":"https://github.com/Shopify/camus","homepage":null,"size":42257,"stargazers_count":1,"watchers_count":1,"language":"Java","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":3,"forks":0,"open_issues":3,"watchers":1,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/Shopify/camus/pulls/38"},"html":{"href":"https://github.com/Shopify/camus/pull/38"},"issue":{"href":"https://api.github.com/repos/Shopify/camus/issues/38"},"comments":{"href":"https://api.github.com/repos/Shopify/camus/issues/38/comments"},"review_comments":{"href":"https://api.github.com/repos/Shopify/camus/pulls/38/comments"},"review_comment":{"href":"https://api.github.com/repos/Shopify/camus/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/Shopify/camus/pulls/38/commits"},"statuses":{"href":"https://api.github.com/repos/Shopify/camus/statuses/ce0e853c893321e86425e57fa5b0b57addd7ba04"}}}}