{"action":"created","comment":{"url":"https://api.github.com/repos/kubernetes/kubernetes/pulls/comments/46180749","id":46180749,"diff_hunk":"@@ -0,0 +1,66 @@\n+<!-- BEGIN MUNGE: UNVERSIONED_WARNING -->\n+\n+<!-- BEGIN STRIP_FOR_RELEASE -->\n+\n+<img src=\"http://kubernetes.io/img/warning.png\" alt=\"WARNING\"\n+     width=\"25\" height=\"25\">\n+<img src=\"http://kubernetes.io/img/warning.png\" alt=\"WARNING\"\n+     width=\"25\" height=\"25\">\n+<img src=\"http://kubernetes.io/img/warning.png\" alt=\"WARNING\"\n+     width=\"25\" height=\"25\">\n+<img src=\"http://kubernetes.io/img/warning.png\" alt=\"WARNING\"\n+     width=\"25\" height=\"25\">\n+<img src=\"http://kubernetes.io/img/warning.png\" alt=\"WARNING\"\n+     width=\"25\" height=\"25\">\n+\n+<h2>PLEASE NOTE: This document applies to the HEAD of the source tree</h2>\n+\n+If you are using a released version of Kubernetes, you should\n+refer to the docs that go with that version.\n+\n+<strong>\n+The latest release of this document can be found\n+[here](http://releases.k8s.io/release-1.1/docs/devel/testing.md).\n+\n+Documentation for other releases can be found at\n+[releases.k8s.io](http://releases.k8s.io).\n+</strong>\n+--\n+\n+<!-- END STRIP_FOR_RELEASE -->\n+\n+<!-- END MUNGE: UNVERSIONED_WARNING -->\n+Kubernetes Commit Queue Testing\n+===============================\n+\n+A quick overview of how we add, remove and recycle tests from CI.\n+\n+## What is CI?\n+\n+Throughout this document we will refer to CI as any suite of e2e tests that can potentially hold up the submit queue, this means Kubernetes PRs must pass these tests prior to getting merged.\n+\n+## Adding a test to CI\n+\n+When first adding a test it should *not* go straight into CI, because failures block ordinary development. A test should only be added to CI after is has been running in some non-CI suite long enought to establish a track record showing that the test does not fail when run against *working* software. A suite named `flaky` exists, and can be overloaded to mean `experimental` and used for this reason (can it really?). In addition to this track record, consider the following as requirements:","path":"docs/devel/testing.md","position":44,"original_position":44,"commit_id":"d256f4d92b85b45ead32d9e18d5305770d2a68c2","original_commit_id":"d256f4d92b85b45ead32d9e18d5305770d2a68c2","user":{"login":"bprashanth","id":10927820,"avatar_url":"https://avatars.githubusercontent.com/u/10927820?v=3","gravatar_id":"","url":"https://api.github.com/users/bprashanth","html_url":"https://github.com/bprashanth","followers_url":"https://api.github.com/users/bprashanth/followers","following_url":"https://api.github.com/users/bprashanth/following{/other_user}","gists_url":"https://api.github.com/users/bprashanth/gists{/gist_id}","starred_url":"https://api.github.com/users/bprashanth/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bprashanth/subscriptions","organizations_url":"https://api.github.com/users/bprashanth/orgs","repos_url":"https://api.github.com/users/bprashanth/repos","events_url":"https://api.github.com/users/bprashanth/events{/privacy}","received_events_url":"https://api.github.com/users/bprashanth/received_events","type":"User","site_admin":false},"body":"Since we don't have a way to selectively hold up a subtree yet, I think we should make a call as to which is more important. If we ran volume tests out of CI people who care about the node can still retro actively revert things, no?","created_at":"2015-11-30T18:20:07Z","updated_at":"2015-11-30T18:20:07Z","html_url":"https://github.com/kubernetes/kubernetes/pull/17791#discussion_r46180749","pull_request_url":"https://api.github.com/repos/kubernetes/kubernetes/pulls/17791","_links":{"self":{"href":"https://api.github.com/repos/kubernetes/kubernetes/pulls/comments/46180749"},"html":{"href":"https://github.com/kubernetes/kubernetes/pull/17791#discussion_r46180749"},"pull_request":{"href":"https://api.github.com/repos/kubernetes/kubernetes/pulls/17791"}}},"pull_request":{"url":"https://api.github.com/repos/kubernetes/kubernetes/pulls/17791","id":51833762,"html_url":"https://github.com/kubernetes/kubernetes/pull/17791","diff_url":"https://github.com/kubernetes/kubernetes/pull/17791.diff","patch_url":"https://github.com/kubernetes/kubernetes/pull/17791.patch","issue_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/17791","number":17791,"state":"open","locked":false,"title":"[wip] CI testing guidelines","user":{"login":"bprashanth","id":10927820,"avatar_url":"https://avatars.githubusercontent.com/u/10927820?v=3","gravatar_id":"","url":"https://api.github.com/users/bprashanth","html_url":"https://github.com/bprashanth","followers_url":"https://api.github.com/users/bprashanth/followers","following_url":"https://api.github.com/users/bprashanth/following{/other_user}","gists_url":"https://api.github.com/users/bprashanth/gists{/gist_id}","starred_url":"https://api.github.com/users/bprashanth/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bprashanth/subscriptions","organizations_url":"https://api.github.com/users/bprashanth/orgs","repos_url":"https://api.github.com/users/bprashanth/repos","events_url":"https://api.github.com/users/bprashanth/events{/privacy}","received_events_url":"https://api.github.com/users/bprashanth/received_events","type":"User","site_admin":false},"body":"Looking for opinions. Assigning to Jeff pseudo randomly, because he usually has sage thoughts on testing. ","created_at":"2015-11-25T19:00:31Z","updated_at":"2015-11-30T18:20:07Z","closed_at":null,"merged_at":null,"merge_commit_sha":"583e0d8ac80dd87e5f8272f89eab9d7c30f2d6dd","assignee":{"login":"ihmccreery","id":368416,"avatar_url":"https://avatars.githubusercontent.com/u/368416?v=3","gravatar_id":"","url":"https://api.github.com/users/ihmccreery","html_url":"https://github.com/ihmccreery","followers_url":"https://api.github.com/users/ihmccreery/followers","following_url":"https://api.github.com/users/ihmccreery/following{/other_user}","gists_url":"https://api.github.com/users/ihmccreery/gists{/gist_id}","starred_url":"https://api.github.com/users/ihmccreery/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ihmccreery/subscriptions","organizations_url":"https://api.github.com/users/ihmccreery/orgs","repos_url":"https://api.github.com/users/ihmccreery/repos","events_url":"https://api.github.com/users/ihmccreery/events{/privacy}","received_events_url":"https://api.github.com/users/ihmccreery/received_events","type":"User","site_admin":false},"milestone":null,"commits_url":"https://api.github.com/repos/kubernetes/kubernetes/pulls/17791/commits","review_comments_url":"https://api.github.com/repos/kubernetes/kubernetes/pulls/17791/comments","review_comment_url":"https://api.github.com/repos/kubernetes/kubernetes/pulls/comments{/number}","comments_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/17791/comments","statuses_url":"https://api.github.com/repos/kubernetes/kubernetes/statuses/d256f4d92b85b45ead32d9e18d5305770d2a68c2","head":{"label":"bprashanth:ci_doc","ref":"ci_doc","sha":"d256f4d92b85b45ead32d9e18d5305770d2a68c2","user":{"login":"bprashanth","id":10927820,"avatar_url":"https://avatars.githubusercontent.com/u/10927820?v=3","gravatar_id":"","url":"https://api.github.com/users/bprashanth","html_url":"https://github.com/bprashanth","followers_url":"https://api.github.com/users/bprashanth/followers","following_url":"https://api.github.com/users/bprashanth/following{/other_user}","gists_url":"https://api.github.com/users/bprashanth/gists{/gist_id}","starred_url":"https://api.github.com/users/bprashanth/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bprashanth/subscriptions","organizations_url":"https://api.github.com/users/bprashanth/orgs","repos_url":"https://api.github.com/users/bprashanth/repos","events_url":"https://api.github.com/users/bprashanth/events{/privacy}","received_events_url":"https://api.github.com/users/bprashanth/received_events","type":"User","site_admin":false},"repo":{"id":30552514,"name":"kubernetes","full_name":"bprashanth/kubernetes","owner":{"login":"bprashanth","id":10927820,"avatar_url":"https://avatars.githubusercontent.com/u/10927820?v=3","gravatar_id":"","url":"https://api.github.com/users/bprashanth","html_url":"https://github.com/bprashanth","followers_url":"https://api.github.com/users/bprashanth/followers","following_url":"https://api.github.com/users/bprashanth/following{/other_user}","gists_url":"https://api.github.com/users/bprashanth/gists{/gist_id}","starred_url":"https://api.github.com/users/bprashanth/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bprashanth/subscriptions","organizations_url":"https://api.github.com/users/bprashanth/orgs","repos_url":"https://api.github.com/users/bprashanth/repos","events_url":"https://api.github.com/users/bprashanth/events{/privacy}","received_events_url":"https://api.github.com/users/bprashanth/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/bprashanth/kubernetes","description":"Container Cluster Manager","fork":true,"url":"https://api.github.com/repos/bprashanth/kubernetes","forks_url":"https://api.github.com/repos/bprashanth/kubernetes/forks","keys_url":"https://api.github.com/repos/bprashanth/kubernetes/keys{/key_id}","collaborators_url":"https://api.github.com/repos/bprashanth/kubernetes/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/bprashanth/kubernetes/teams","hooks_url":"https://api.github.com/repos/bprashanth/kubernetes/hooks","issue_events_url":"https://api.github.com/repos/bprashanth/kubernetes/issues/events{/number}","events_url":"https://api.github.com/repos/bprashanth/kubernetes/events","assignees_url":"https://api.github.com/repos/bprashanth/kubernetes/assignees{/user}","branches_url":"https://api.github.com/repos/bprashanth/kubernetes/branches{/branch}","tags_url":"https://api.github.com/repos/bprashanth/kubernetes/tags","blobs_url":"https://api.github.com/repos/bprashanth/kubernetes/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/bprashanth/kubernetes/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/bprashanth/kubernetes/git/refs{/sha}","trees_url":"https://api.github.com/repos/bprashanth/kubernetes/git/trees{/sha}","statuses_url":"https://api.github.com/repos/bprashanth/kubernetes/statuses/{sha}","languages_url":"https://api.github.com/repos/bprashanth/kubernetes/languages","stargazers_url":"https://api.github.com/repos/bprashanth/kubernetes/stargazers","contributors_url":"https://api.github.com/repos/bprashanth/kubernetes/contributors","subscribers_url":"https://api.github.com/repos/bprashanth/kubernetes/subscribers","subscription_url":"https://api.github.com/repos/bprashanth/kubernetes/subscription","commits_url":"https://api.github.com/repos/bprashanth/kubernetes/commits{/sha}","git_commits_url":"https://api.github.com/repos/bprashanth/kubernetes/git/commits{/sha}","comments_url":"https://api.github.com/repos/bprashanth/kubernetes/comments{/number}","issue_comment_url":"https://api.github.com/repos/bprashanth/kubernetes/issues/comments{/number}","contents_url":"https://api.github.com/repos/bprashanth/kubernetes/contents/{+path}","compare_url":"https://api.github.com/repos/bprashanth/kubernetes/compare/{base}...{head}","merges_url":"https://api.github.com/repos/bprashanth/kubernetes/merges","archive_url":"https://api.github.com/repos/bprashanth/kubernetes/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/bprashanth/kubernetes/downloads","issues_url":"https://api.github.com/repos/bprashanth/kubernetes/issues{/number}","pulls_url":"https://api.github.com/repos/bprashanth/kubernetes/pulls{/number}","milestones_url":"https://api.github.com/repos/bprashanth/kubernetes/milestones{/number}","notifications_url":"https://api.github.com/repos/bprashanth/kubernetes/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/bprashanth/kubernetes/labels{/name}","releases_url":"https://api.github.com/repos/bprashanth/kubernetes/releases{/id}","created_at":"2015-02-09T18:58:38Z","updated_at":"2015-02-09T18:58:42Z","pushed_at":"2015-11-30T17:10:43Z","git_url":"git://github.com/bprashanth/kubernetes.git","ssh_url":"git@github.com:bprashanth/kubernetes.git","clone_url":"https://github.com/bprashanth/kubernetes.git","svn_url":"https://github.com/bprashanth/kubernetes","homepage":"http://kubernetes.io","size":140146,"stargazers_count":0,"watchers_count":0,"language":"Go","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":true,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"kubernetes:master","ref":"master","sha":"581a1ad1af7acb1aab08ad178d0004753ea15fbd","user":{"login":"kubernetes","id":13629408,"avatar_url":"https://avatars.githubusercontent.com/u/13629408?v=3","gravatar_id":"","url":"https://api.github.com/users/kubernetes","html_url":"https://github.com/kubernetes","followers_url":"https://api.github.com/users/kubernetes/followers","following_url":"https://api.github.com/users/kubernetes/following{/other_user}","gists_url":"https://api.github.com/users/kubernetes/gists{/gist_id}","starred_url":"https://api.github.com/users/kubernetes/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kubernetes/subscriptions","organizations_url":"https://api.github.com/users/kubernetes/orgs","repos_url":"https://api.github.com/users/kubernetes/repos","events_url":"https://api.github.com/users/kubernetes/events{/privacy}","received_events_url":"https://api.github.com/users/kubernetes/received_events","type":"Organization","site_admin":false},"repo":{"id":20580498,"name":"kubernetes","full_name":"kubernetes/kubernetes","owner":{"login":"kubernetes","id":13629408,"avatar_url":"https://avatars.githubusercontent.com/u/13629408?v=3","gravatar_id":"","url":"https://api.github.com/users/kubernetes","html_url":"https://github.com/kubernetes","followers_url":"https://api.github.com/users/kubernetes/followers","following_url":"https://api.github.com/users/kubernetes/following{/other_user}","gists_url":"https://api.github.com/users/kubernetes/gists{/gist_id}","starred_url":"https://api.github.com/users/kubernetes/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kubernetes/subscriptions","organizations_url":"https://api.github.com/users/kubernetes/orgs","repos_url":"https://api.github.com/users/kubernetes/repos","events_url":"https://api.github.com/users/kubernetes/events{/privacy}","received_events_url":"https://api.github.com/users/kubernetes/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/kubernetes/kubernetes","description":"Container Cluster Manager from Google","fork":false,"url":"https://api.github.com/repos/kubernetes/kubernetes","forks_url":"https://api.github.com/repos/kubernetes/kubernetes/forks","keys_url":"https://api.github.com/repos/kubernetes/kubernetes/keys{/key_id}","collaborators_url":"https://api.github.com/repos/kubernetes/kubernetes/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/kubernetes/kubernetes/teams","hooks_url":"https://api.github.com/repos/kubernetes/kubernetes/hooks","issue_events_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/events{/number}","events_url":"https://api.github.com/repos/kubernetes/kubernetes/events","assignees_url":"https://api.github.com/repos/kubernetes/kubernetes/assignees{/user}","branches_url":"https://api.github.com/repos/kubernetes/kubernetes/branches{/branch}","tags_url":"https://api.github.com/repos/kubernetes/kubernetes/tags","blobs_url":"https://api.github.com/repos/kubernetes/kubernetes/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/kubernetes/kubernetes/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/kubernetes/kubernetes/git/refs{/sha}","trees_url":"https://api.github.com/repos/kubernetes/kubernetes/git/trees{/sha}","statuses_url":"https://api.github.com/repos/kubernetes/kubernetes/statuses/{sha}","languages_url":"https://api.github.com/repos/kubernetes/kubernetes/languages","stargazers_url":"https://api.github.com/repos/kubernetes/kubernetes/stargazers","contributors_url":"https://api.github.com/repos/kubernetes/kubernetes/contributors","subscribers_url":"https://api.github.com/repos/kubernetes/kubernetes/subscribers","subscription_url":"https://api.github.com/repos/kubernetes/kubernetes/subscription","commits_url":"https://api.github.com/repos/kubernetes/kubernetes/commits{/sha}","git_commits_url":"https://api.github.com/repos/kubernetes/kubernetes/git/commits{/sha}","comments_url":"https://api.github.com/repos/kubernetes/kubernetes/comments{/number}","issue_comment_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/comments{/number}","contents_url":"https://api.github.com/repos/kubernetes/kubernetes/contents/{+path}","compare_url":"https://api.github.com/repos/kubernetes/kubernetes/compare/{base}...{head}","merges_url":"https://api.github.com/repos/kubernetes/kubernetes/merges","archive_url":"https://api.github.com/repos/kubernetes/kubernetes/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/kubernetes/kubernetes/downloads","issues_url":"https://api.github.com/repos/kubernetes/kubernetes/issues{/number}","pulls_url":"https://api.github.com/repos/kubernetes/kubernetes/pulls{/number}","milestones_url":"https://api.github.com/repos/kubernetes/kubernetes/milestones{/number}","notifications_url":"https://api.github.com/repos/kubernetes/kubernetes/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/kubernetes/kubernetes/labels{/name}","releases_url":"https://api.github.com/repos/kubernetes/kubernetes/releases{/id}","created_at":"2014-06-06T22:56:04Z","updated_at":"2015-11-30T16:08:48Z","pushed_at":"2015-11-30T17:35:23Z","git_url":"git://github.com/kubernetes/kubernetes.git","ssh_url":"git@github.com:kubernetes/kubernetes.git","clone_url":"https://github.com/kubernetes/kubernetes.git","svn_url":"https://github.com/kubernetes/kubernetes","homepage":"http://kubernetes.io","size":291038,"stargazers_count":11533,"watchers_count":11533,"language":"Go","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":true,"forks_count":3099,"mirror_url":null,"open_issues_count":2461,"forks":3099,"open_issues":2461,"watchers":11533,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/kubernetes/kubernetes/pulls/17791"},"html":{"href":"https://github.com/kubernetes/kubernetes/pull/17791"},"issue":{"href":"https://api.github.com/repos/kubernetes/kubernetes/issues/17791"},"comments":{"href":"https://api.github.com/repos/kubernetes/kubernetes/issues/17791/comments"},"review_comments":{"href":"https://api.github.com/repos/kubernetes/kubernetes/pulls/17791/comments"},"review_comment":{"href":"https://api.github.com/repos/kubernetes/kubernetes/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/kubernetes/kubernetes/pulls/17791/commits"},"statuses":{"href":"https://api.github.com/repos/kubernetes/kubernetes/statuses/d256f4d92b85b45ead32d9e18d5305770d2a68c2"}}}}