{"action":"created","comment":{"url":"https://api.github.com/repos/code-dot-org/code-dot-org/pulls/comments/41293116","id":41293116,"diff_hunk":"@@ -0,0 +1,80 @@\n+require 'fakeweb'\n+require 'minitest/autorun'\n+require 'rack/test'\n+require_relative '../../deployment'\n+require_relative '../../lib/cdo/hip_chat'\n+\n+class HipchatTest < Minitest::Test\n+\n+  MAX_WAIT_SECONDS = 10\n+\n+  # Use a short retry backoff for this test to keep the test fast.\n+  BACKOFF = 0.1\n+\n+  def setup\n+    CDO.hip_chat_logging = true\n+    CDO.log.level = 5  # Log only fatal exceptions to avoid test spew.","path":"shared/test/test_hip_chat.rb","position":null,"original_position":16,"commit_id":"8de5458167a941968aa8a453fa6da1a089679486","original_commit_id":"1ab4b67e433db864a7d9414020b75dbc99699343","user":{"login":"philbogle","id":5198586,"avatar_url":"https://avatars.githubusercontent.com/u/5198586?v=3","gravatar_id":"","url":"https://api.github.com/users/philbogle","html_url":"https://github.com/philbogle","followers_url":"https://api.github.com/users/philbogle/followers","following_url":"https://api.github.com/users/philbogle/following{/other_user}","gists_url":"https://api.github.com/users/philbogle/gists{/gist_id}","starred_url":"https://api.github.com/users/philbogle/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/philbogle/subscriptions","organizations_url":"https://api.github.com/users/philbogle/orgs","repos_url":"https://api.github.com/users/philbogle/repos","events_url":"https://api.github.com/users/philbogle/events{/privacy}","received_events_url":"https://api.github.com/users/philbogle/received_events","type":"User","site_admin":false},"body":"Done.","created_at":"2015-10-06T17:13:08Z","updated_at":"2015-10-06T17:13:08Z","html_url":"https://github.com/code-dot-org/code-dot-org/pull/4358#discussion_r41293116","pull_request_url":"https://api.github.com/repos/code-dot-org/code-dot-org/pulls/4358","_links":{"self":{"href":"https://api.github.com/repos/code-dot-org/code-dot-org/pulls/comments/41293116"},"html":{"href":"https://github.com/code-dot-org/code-dot-org/pull/4358#discussion_r41293116"},"pull_request":{"href":"https://api.github.com/repos/code-dot-org/code-dot-org/pulls/4358"}}},"pull_request":{"url":"https://api.github.com/repos/code-dot-org/code-dot-org/pulls/4358","id":46863267,"html_url":"https://github.com/code-dot-org/code-dot-org/pull/4358","diff_url":"https://github.com/code-dot-org/code-dot-org/pull/4358.diff","patch_url":"https://github.com/code-dot-org/code-dot-org/pull/4358.patch","issue_url":"https://api.github.com/repos/code-dot-org/code-dot-org/issues/4358","number":4358,"state":"open","locked":false,"title":"Add exponential back-off to HipChat REST requests","user":{"login":"philbogle","id":5198586,"avatar_url":"https://avatars.githubusercontent.com/u/5198586?v=3","gravatar_id":"","url":"https://api.github.com/users/philbogle","html_url":"https://github.com/philbogle","followers_url":"https://api.github.com/users/philbogle/followers","following_url":"https://api.github.com/users/philbogle/following{/other_user}","gists_url":"https://api.github.com/users/philbogle/gists{/gist_id}","starred_url":"https://api.github.com/users/philbogle/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/philbogle/subscriptions","organizations_url":"https://api.github.com/users/philbogle/orgs","repos_url":"https://api.github.com/users/philbogle/repos","events_url":"https://api.github.com/users/philbogle/events{/privacy}","received_events_url":"https://api.github.com/users/philbogle/received_events","type":"User","site_admin":false},"body":"Previously, if a HipChat REST request failed, we were giving up, result in lost messages on any failed request.\r\n\r\nNow, we retry with exponential backoff up to three times, at which point we give up and log to the CDO.logger.  (We don't want to retry indefinitely even with exponential backoff since this could lead us to OOM if Hipchat is unavailable.)  \r\n\r\nThis is intended to help with transient hipchat server errors;  it is not intended to help with the case when we are exceeding the max HipChat message rate. (That will be addressed separately by Laurel's change to reduce success logging rates and perhaps by changing to an auth token that permits higher rates than our current user token.)\r\n\r\nThe waiting and retries (but not the original request, to preserve current behavior) are done on a new thread to avoid stalling the main thread.  HipChat logging is a low frequency event and is only done in staging/test, so occasionally starting a thread after a HipChat failure is acceptable.  \r\n\r\nWe use a thread rather than asynchronous HTTP to simplify the code-- the built-in Ruby HTTP libraries don't even support asynchronous HTTP, and the exponential backoff logic with asynchronous HTTP is quite a bit trickier.\r\n\r\nThis PR also adds unit tests of Hipchat logging to verify that the exponential backoff behavior is correctly working.","created_at":"2015-10-06T05:13:02Z","updated_at":"2015-10-06T17:13:08Z","closed_at":null,"merged_at":null,"merge_commit_sha":"3fc7b5b9308a7749ccdc80c1f0a7bcf0d66bc815","assignee":{"login":"trevorb","id":17502,"avatar_url":"https://avatars.githubusercontent.com/u/17502?v=3","gravatar_id":"","url":"https://api.github.com/users/trevorb","html_url":"https://github.com/trevorb","followers_url":"https://api.github.com/users/trevorb/followers","following_url":"https://api.github.com/users/trevorb/following{/other_user}","gists_url":"https://api.github.com/users/trevorb/gists{/gist_id}","starred_url":"https://api.github.com/users/trevorb/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/trevorb/subscriptions","organizations_url":"https://api.github.com/users/trevorb/orgs","repos_url":"https://api.github.com/users/trevorb/repos","events_url":"https://api.github.com/users/trevorb/events{/privacy}","received_events_url":"https://api.github.com/users/trevorb/received_events","type":"User","site_admin":false},"milestone":null,"commits_url":"https://api.github.com/repos/code-dot-org/code-dot-org/pulls/4358/commits","review_comments_url":"https://api.github.com/repos/code-dot-org/code-dot-org/pulls/4358/comments","review_comment_url":"https://api.github.com/repos/code-dot-org/code-dot-org/pulls/comments{/number}","comments_url":"https://api.github.com/repos/code-dot-org/code-dot-org/issues/4358/comments","statuses_url":"https://api.github.com/repos/code-dot-org/code-dot-org/statuses/8de5458167a941968aa8a453fa6da1a089679486","head":{"label":"code-dot-org:hipchat","ref":"hipchat","sha":"8de5458167a941968aa8a453fa6da1a089679486","user":{"login":"code-dot-org","id":5124627,"avatar_url":"https://avatars.githubusercontent.com/u/5124627?v=3","gravatar_id":"","url":"https://api.github.com/users/code-dot-org","html_url":"https://github.com/code-dot-org","followers_url":"https://api.github.com/users/code-dot-org/followers","following_url":"https://api.github.com/users/code-dot-org/following{/other_user}","gists_url":"https://api.github.com/users/code-dot-org/gists{/gist_id}","starred_url":"https://api.github.com/users/code-dot-org/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/code-dot-org/subscriptions","organizations_url":"https://api.github.com/users/code-dot-org/orgs","repos_url":"https://api.github.com/users/code-dot-org/repos","events_url":"https://api.github.com/users/code-dot-org/events{/privacy}","received_events_url":"https://api.github.com/users/code-dot-org/received_events","type":"Organization","site_admin":false},"repo":{"id":24250009,"name":"code-dot-org","full_name":"code-dot-org/code-dot-org","owner":{"login":"code-dot-org","id":5124627,"avatar_url":"https://avatars.githubusercontent.com/u/5124627?v=3","gravatar_id":"","url":"https://api.github.com/users/code-dot-org","html_url":"https://github.com/code-dot-org","followers_url":"https://api.github.com/users/code-dot-org/followers","following_url":"https://api.github.com/users/code-dot-org/following{/other_user}","gists_url":"https://api.github.com/users/code-dot-org/gists{/gist_id}","starred_url":"https://api.github.com/users/code-dot-org/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/code-dot-org/subscriptions","organizations_url":"https://api.github.com/users/code-dot-org/orgs","repos_url":"https://api.github.com/users/code-dot-org/repos","events_url":"https://api.github.com/users/code-dot-org/events{/privacy}","received_events_url":"https://api.github.com/users/code-dot-org/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/code-dot-org/code-dot-org","description":"The code powering code.org and studio.code.org","fork":false,"url":"https://api.github.com/repos/code-dot-org/code-dot-org","forks_url":"https://api.github.com/repos/code-dot-org/code-dot-org/forks","keys_url":"https://api.github.com/repos/code-dot-org/code-dot-org/keys{/key_id}","collaborators_url":"https://api.github.com/repos/code-dot-org/code-dot-org/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/code-dot-org/code-dot-org/teams","hooks_url":"https://api.github.com/repos/code-dot-org/code-dot-org/hooks","issue_events_url":"https://api.github.com/repos/code-dot-org/code-dot-org/issues/events{/number}","events_url":"https://api.github.com/repos/code-dot-org/code-dot-org/events","assignees_url":"https://api.github.com/repos/code-dot-org/code-dot-org/assignees{/user}","branches_url":"https://api.github.com/repos/code-dot-org/code-dot-org/branches{/branch}","tags_url":"https://api.github.com/repos/code-dot-org/code-dot-org/tags","blobs_url":"https://api.github.com/repos/code-dot-org/code-dot-org/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/code-dot-org/code-dot-org/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/code-dot-org/code-dot-org/git/refs{/sha}","trees_url":"https://api.github.com/repos/code-dot-org/code-dot-org/git/trees{/sha}","statuses_url":"https://api.github.com/repos/code-dot-org/code-dot-org/statuses/{sha}","languages_url":"https://api.github.com/repos/code-dot-org/code-dot-org/languages","stargazers_url":"https://api.github.com/repos/code-dot-org/code-dot-org/stargazers","contributors_url":"https://api.github.com/repos/code-dot-org/code-dot-org/contributors","subscribers_url":"https://api.github.com/repos/code-dot-org/code-dot-org/subscribers","subscription_url":"https://api.github.com/repos/code-dot-org/code-dot-org/subscription","commits_url":"https://api.github.com/repos/code-dot-org/code-dot-org/commits{/sha}","git_commits_url":"https://api.github.com/repos/code-dot-org/code-dot-org/git/commits{/sha}","comments_url":"https://api.github.com/repos/code-dot-org/code-dot-org/comments{/number}","issue_comment_url":"https://api.github.com/repos/code-dot-org/code-dot-org/issues/comments{/number}","contents_url":"https://api.github.com/repos/code-dot-org/code-dot-org/contents/{+path}","compare_url":"https://api.github.com/repos/code-dot-org/code-dot-org/compare/{base}...{head}","merges_url":"https://api.github.com/repos/code-dot-org/code-dot-org/merges","archive_url":"https://api.github.com/repos/code-dot-org/code-dot-org/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/code-dot-org/code-dot-org/downloads","issues_url":"https://api.github.com/repos/code-dot-org/code-dot-org/issues{/number}","pulls_url":"https://api.github.com/repos/code-dot-org/code-dot-org/pulls{/number}","milestones_url":"https://api.github.com/repos/code-dot-org/code-dot-org/milestones{/number}","notifications_url":"https://api.github.com/repos/code-dot-org/code-dot-org/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/code-dot-org/code-dot-org/labels{/name}","releases_url":"https://api.github.com/repos/code-dot-org/code-dot-org/releases{/id}","created_at":"2014-09-20T00:46:11Z","updated_at":"2015-09-16T14:46:10Z","pushed_at":"2015-10-06T17:12:22Z","git_url":"git://github.com/code-dot-org/code-dot-org.git","ssh_url":"git@github.com:code-dot-org/code-dot-org.git","clone_url":"https://github.com/code-dot-org/code-dot-org.git","svn_url":"https://github.com/code-dot-org/code-dot-org","homepage":"http://code.org","size":2844433,"stargazers_count":58,"watchers_count":58,"language":"JavaScript","has_issues":false,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":39,"mirror_url":null,"open_issues_count":47,"forks":39,"open_issues":47,"watchers":58,"default_branch":"staging"}},"base":{"label":"code-dot-org:staging","ref":"staging","sha":"64b1f802bafd7e4af193edcef2136f7ea36bf46d","user":{"login":"code-dot-org","id":5124627,"avatar_url":"https://avatars.githubusercontent.com/u/5124627?v=3","gravatar_id":"","url":"https://api.github.com/users/code-dot-org","html_url":"https://github.com/code-dot-org","followers_url":"https://api.github.com/users/code-dot-org/followers","following_url":"https://api.github.com/users/code-dot-org/following{/other_user}","gists_url":"https://api.github.com/users/code-dot-org/gists{/gist_id}","starred_url":"https://api.github.com/users/code-dot-org/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/code-dot-org/subscriptions","organizations_url":"https://api.github.com/users/code-dot-org/orgs","repos_url":"https://api.github.com/users/code-dot-org/repos","events_url":"https://api.github.com/users/code-dot-org/events{/privacy}","received_events_url":"https://api.github.com/users/code-dot-org/received_events","type":"Organization","site_admin":false},"repo":{"id":24250009,"name":"code-dot-org","full_name":"code-dot-org/code-dot-org","owner":{"login":"code-dot-org","id":5124627,"avatar_url":"https://avatars.githubusercontent.com/u/5124627?v=3","gravatar_id":"","url":"https://api.github.com/users/code-dot-org","html_url":"https://github.com/code-dot-org","followers_url":"https://api.github.com/users/code-dot-org/followers","following_url":"https://api.github.com/users/code-dot-org/following{/other_user}","gists_url":"https://api.github.com/users/code-dot-org/gists{/gist_id}","starred_url":"https://api.github.com/users/code-dot-org/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/code-dot-org/subscriptions","organizations_url":"https://api.github.com/users/code-dot-org/orgs","repos_url":"https://api.github.com/users/code-dot-org/repos","events_url":"https://api.github.com/users/code-dot-org/events{/privacy}","received_events_url":"https://api.github.com/users/code-dot-org/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/code-dot-org/code-dot-org","description":"The code powering code.org and studio.code.org","fork":false,"url":"https://api.github.com/repos/code-dot-org/code-dot-org","forks_url":"https://api.github.com/repos/code-dot-org/code-dot-org/forks","keys_url":"https://api.github.com/repos/code-dot-org/code-dot-org/keys{/key_id}","collaborators_url":"https://api.github.com/repos/code-dot-org/code-dot-org/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/code-dot-org/code-dot-org/teams","hooks_url":"https://api.github.com/repos/code-dot-org/code-dot-org/hooks","issue_events_url":"https://api.github.com/repos/code-dot-org/code-dot-org/issues/events{/number}","events_url":"https://api.github.com/repos/code-dot-org/code-dot-org/events","assignees_url":"https://api.github.com/repos/code-dot-org/code-dot-org/assignees{/user}","branches_url":"https://api.github.com/repos/code-dot-org/code-dot-org/branches{/branch}","tags_url":"https://api.github.com/repos/code-dot-org/code-dot-org/tags","blobs_url":"https://api.github.com/repos/code-dot-org/code-dot-org/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/code-dot-org/code-dot-org/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/code-dot-org/code-dot-org/git/refs{/sha}","trees_url":"https://api.github.com/repos/code-dot-org/code-dot-org/git/trees{/sha}","statuses_url":"https://api.github.com/repos/code-dot-org/code-dot-org/statuses/{sha}","languages_url":"https://api.github.com/repos/code-dot-org/code-dot-org/languages","stargazers_url":"https://api.github.com/repos/code-dot-org/code-dot-org/stargazers","contributors_url":"https://api.github.com/repos/code-dot-org/code-dot-org/contributors","subscribers_url":"https://api.github.com/repos/code-dot-org/code-dot-org/subscribers","subscription_url":"https://api.github.com/repos/code-dot-org/code-dot-org/subscription","commits_url":"https://api.github.com/repos/code-dot-org/code-dot-org/commits{/sha}","git_commits_url":"https://api.github.com/repos/code-dot-org/code-dot-org/git/commits{/sha}","comments_url":"https://api.github.com/repos/code-dot-org/code-dot-org/comments{/number}","issue_comment_url":"https://api.github.com/repos/code-dot-org/code-dot-org/issues/comments{/number}","contents_url":"https://api.github.com/repos/code-dot-org/code-dot-org/contents/{+path}","compare_url":"https://api.github.com/repos/code-dot-org/code-dot-org/compare/{base}...{head}","merges_url":"https://api.github.com/repos/code-dot-org/code-dot-org/merges","archive_url":"https://api.github.com/repos/code-dot-org/code-dot-org/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/code-dot-org/code-dot-org/downloads","issues_url":"https://api.github.com/repos/code-dot-org/code-dot-org/issues{/number}","pulls_url":"https://api.github.com/repos/code-dot-org/code-dot-org/pulls{/number}","milestones_url":"https://api.github.com/repos/code-dot-org/code-dot-org/milestones{/number}","notifications_url":"https://api.github.com/repos/code-dot-org/code-dot-org/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/code-dot-org/code-dot-org/labels{/name}","releases_url":"https://api.github.com/repos/code-dot-org/code-dot-org/releases{/id}","created_at":"2014-09-20T00:46:11Z","updated_at":"2015-09-16T14:46:10Z","pushed_at":"2015-10-06T17:12:22Z","git_url":"git://github.com/code-dot-org/code-dot-org.git","ssh_url":"git@github.com:code-dot-org/code-dot-org.git","clone_url":"https://github.com/code-dot-org/code-dot-org.git","svn_url":"https://github.com/code-dot-org/code-dot-org","homepage":"http://code.org","size":2844433,"stargazers_count":58,"watchers_count":58,"language":"JavaScript","has_issues":false,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":39,"mirror_url":null,"open_issues_count":47,"forks":39,"open_issues":47,"watchers":58,"default_branch":"staging"}},"_links":{"self":{"href":"https://api.github.com/repos/code-dot-org/code-dot-org/pulls/4358"},"html":{"href":"https://github.com/code-dot-org/code-dot-org/pull/4358"},"issue":{"href":"https://api.github.com/repos/code-dot-org/code-dot-org/issues/4358"},"comments":{"href":"https://api.github.com/repos/code-dot-org/code-dot-org/issues/4358/comments"},"review_comments":{"href":"https://api.github.com/repos/code-dot-org/code-dot-org/pulls/4358/comments"},"review_comment":{"href":"https://api.github.com/repos/code-dot-org/code-dot-org/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/code-dot-org/code-dot-org/pulls/4358/commits"},"statuses":{"href":"https://api.github.com/repos/code-dot-org/code-dot-org/statuses/8de5458167a941968aa8a453fa6da1a089679486"}}}}