{"action":"created","comment":{"url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/pulls/comments/41297226","id":41297226,"diff_hunk":"@@ -498,6 +498,75 @@ Flag | Notes\n `--https_proxy`      | Needed for package manager or Ubuntu guest and for from github downloaded packages\n `--ftp_proxy`       | Needed for some Perfkit packages\n \n+CONFIGS AND USER CONFIGS\n+==================\n+Each benchmark now has an independent configuration which is written in YAML.\n+A user config is a way to override this default configuration on a per benchmark\n+basis. This allows for much more complex setups than previously possible,\n+including running benchmarks across clouds.\n+\n+A benchmark configuration has a somewhat simple structure. It is essentially\n+just a series of nested dictionaries. At the top level, it contains VM groups.\n+VM groups are logical groups of homogenous machines. The VM groups hold both a\n+\"vm_spec\" and a \"disk_spec\" which contain the parameters needed to\n+create members of that group. Here is an example of an expanded configuration:\n+```\n+iperf:\n+  vm_groups:\n+    iperf_vm_1:\n+      vm_count: 1\n+      vm_spec:\n+        GCP:\n+          machine_type: n1-standard-1\n+          image: ubuntu-14-04\n+          zone: us-central1-c\n+        AWS:\n+          machine_type: m3.medium\n+          image: ami-######\n+          zone: us-east-1a\n+        # Other clouds here...\n+    iperf_vm_2:\n+      vm_count: 1\n+      # This specifies the cloud to use for the group. This allows for\n+      benchmark configurations that span clouds.\n+      cloud: AWS\n+      vm_spec:\n+        GCP:\n+          machine_type: n1-standard-1\n+          image: ubuntu-14-04\n+          zone: us-central1-c\n+        # Other clouds here...\n+      disk_count: 0\n+      disk_spec:\n+        GCP:\n+          disk_size: 500\n+          disk_type: standard\n+        # Other clouds here...\n+```\n+\n+For a complete list of keys for VmSpecs and DiskSpecs see\n+virtual_machine.BaseVmSpec and disk.BaseDiskSpec and their derived classes.","path":"README.md","position":51,"original_position":51,"commit_id":"d6e68dbb7e143ece14b0b5e9e30907ab9cebfa2f","original_commit_id":"d6e68dbb7e143ece14b0b5e9e30907ab9cebfa2f","user":{"login":"cmccoy","id":564884,"avatar_url":"https://avatars.githubusercontent.com/u/564884?v=3","gravatar_id":"","url":"https://api.github.com/users/cmccoy","html_url":"https://github.com/cmccoy","followers_url":"https://api.github.com/users/cmccoy/followers","following_url":"https://api.github.com/users/cmccoy/following{/other_user}","gists_url":"https://api.github.com/users/cmccoy/gists{/gist_id}","starred_url":"https://api.github.com/users/cmccoy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cmccoy/subscriptions","organizations_url":"https://api.github.com/users/cmccoy/orgs","repos_url":"https://api.github.com/users/cmccoy/repos","events_url":"https://api.github.com/users/cmccoy/events{/privacy}","received_events_url":"https://api.github.com/users/cmccoy/received_events","type":"User","site_admin":false},"body":"Could you add links?","created_at":"2015-10-06T17:45:43Z","updated_at":"2015-10-06T17:45:43Z","html_url":"https://github.com/GoogleCloudPlatform/PerfKitBenchmarker/pull/510#discussion_r41297226","pull_request_url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/pulls/510","_links":{"self":{"href":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/pulls/comments/41297226"},"html":{"href":"https://github.com/GoogleCloudPlatform/PerfKitBenchmarker/pull/510#discussion_r41297226"},"pull_request":{"href":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/pulls/510"}}},"pull_request":{"url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/pulls/510","id":46670558,"html_url":"https://github.com/GoogleCloudPlatform/PerfKitBenchmarker/pull/510","diff_url":"https://github.com/GoogleCloudPlatform/PerfKitBenchmarker/pull/510.diff","patch_url":"https://github.com/GoogleCloudPlatform/PerfKitBenchmarker/pull/510.patch","issue_url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/issues/510","number":510,"state":"open","locked":false,"title":"Adds configs","user":{"login":"ehankland","id":8193699,"avatar_url":"https://avatars.githubusercontent.com/u/8193699?v=3","gravatar_id":"","url":"https://api.github.com/users/ehankland","html_url":"https://github.com/ehankland","followers_url":"https://api.github.com/users/ehankland/followers","following_url":"https://api.github.com/users/ehankland/following{/other_user}","gists_url":"https://api.github.com/users/ehankland/gists{/gist_id}","starred_url":"https://api.github.com/users/ehankland/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ehankland/subscriptions","organizations_url":"https://api.github.com/users/ehankland/orgs","repos_url":"https://api.github.com/users/ehankland/repos","events_url":"https://api.github.com/users/ehankland/events{/privacy}","received_events_url":"https://api.github.com/users/ehankland/received_events","type":"User","site_admin":false},"body":"","created_at":"2015-10-02T19:07:12Z","updated_at":"2015-10-06T17:45:43Z","closed_at":null,"merged_at":null,"merge_commit_sha":"4fa30b9d545b938c136d2012741a0caec31f03e5","assignee":{"login":"cmccoy","id":564884,"avatar_url":"https://avatars.githubusercontent.com/u/564884?v=3","gravatar_id":"","url":"https://api.github.com/users/cmccoy","html_url":"https://github.com/cmccoy","followers_url":"https://api.github.com/users/cmccoy/followers","following_url":"https://api.github.com/users/cmccoy/following{/other_user}","gists_url":"https://api.github.com/users/cmccoy/gists{/gist_id}","starred_url":"https://api.github.com/users/cmccoy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cmccoy/subscriptions","organizations_url":"https://api.github.com/users/cmccoy/orgs","repos_url":"https://api.github.com/users/cmccoy/repos","events_url":"https://api.github.com/users/cmccoy/events{/privacy}","received_events_url":"https://api.github.com/users/cmccoy/received_events","type":"User","site_admin":false},"milestone":null,"commits_url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/pulls/510/commits","review_comments_url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/pulls/510/comments","review_comment_url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/pulls/comments{/number}","comments_url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/issues/510/comments","statuses_url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/statuses/d6e68dbb7e143ece14b0b5e9e30907ab9cebfa2f","head":{"label":"ehankland:config","ref":"config","sha":"d6e68dbb7e143ece14b0b5e9e30907ab9cebfa2f","user":{"login":"ehankland","id":8193699,"avatar_url":"https://avatars.githubusercontent.com/u/8193699?v=3","gravatar_id":"","url":"https://api.github.com/users/ehankland","html_url":"https://github.com/ehankland","followers_url":"https://api.github.com/users/ehankland/followers","following_url":"https://api.github.com/users/ehankland/following{/other_user}","gists_url":"https://api.github.com/users/ehankland/gists{/gist_id}","starred_url":"https://api.github.com/users/ehankland/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ehankland/subscriptions","organizations_url":"https://api.github.com/users/ehankland/orgs","repos_url":"https://api.github.com/users/ehankland/repos","events_url":"https://api.github.com/users/ehankland/events{/privacy}","received_events_url":"https://api.github.com/users/ehankland/received_events","type":"User","site_admin":false},"repo":{"id":36402001,"name":"PerfKitBenchmarker","full_name":"ehankland/PerfKitBenchmarker","owner":{"login":"ehankland","id":8193699,"avatar_url":"https://avatars.githubusercontent.com/u/8193699?v=3","gravatar_id":"","url":"https://api.github.com/users/ehankland","html_url":"https://github.com/ehankland","followers_url":"https://api.github.com/users/ehankland/followers","following_url":"https://api.github.com/users/ehankland/following{/other_user}","gists_url":"https://api.github.com/users/ehankland/gists{/gist_id}","starred_url":"https://api.github.com/users/ehankland/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ehankland/subscriptions","organizations_url":"https://api.github.com/users/ehankland/orgs","repos_url":"https://api.github.com/users/ehankland/repos","events_url":"https://api.github.com/users/ehankland/events{/privacy}","received_events_url":"https://api.github.com/users/ehankland/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/ehankland/PerfKitBenchmarker","description":"Perfkit Benchmarker contains set of benchmarks to measure and compare cloud offerings. The benchmarks are not tuned (i.e. use defaults) to reflect what most users will see. PerfKit Benchmarker is licensed under the Apache 2 license terms. Please make sure to read, understand and agree to the terms of the LICENSE and CONTRIBUTING files before proceeding.","fork":true,"url":"https://api.github.com/repos/ehankland/PerfKitBenchmarker","forks_url":"https://api.github.com/repos/ehankland/PerfKitBenchmarker/forks","keys_url":"https://api.github.com/repos/ehankland/PerfKitBenchmarker/keys{/key_id}","collaborators_url":"https://api.github.com/repos/ehankland/PerfKitBenchmarker/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/ehankland/PerfKitBenchmarker/teams","hooks_url":"https://api.github.com/repos/ehankland/PerfKitBenchmarker/hooks","issue_events_url":"https://api.github.com/repos/ehankland/PerfKitBenchmarker/issues/events{/number}","events_url":"https://api.github.com/repos/ehankland/PerfKitBenchmarker/events","assignees_url":"https://api.github.com/repos/ehankland/PerfKitBenchmarker/assignees{/user}","branches_url":"https://api.github.com/repos/ehankland/PerfKitBenchmarker/branches{/branch}","tags_url":"https://api.github.com/repos/ehankland/PerfKitBenchmarker/tags","blobs_url":"https://api.github.com/repos/ehankland/PerfKitBenchmarker/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/ehankland/PerfKitBenchmarker/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/ehankland/PerfKitBenchmarker/git/refs{/sha}","trees_url":"https://api.github.com/repos/ehankland/PerfKitBenchmarker/git/trees{/sha}","statuses_url":"https://api.github.com/repos/ehankland/PerfKitBenchmarker/statuses/{sha}","languages_url":"https://api.github.com/repos/ehankland/PerfKitBenchmarker/languages","stargazers_url":"https://api.github.com/repos/ehankland/PerfKitBenchmarker/stargazers","contributors_url":"https://api.github.com/repos/ehankland/PerfKitBenchmarker/contributors","subscribers_url":"https://api.github.com/repos/ehankland/PerfKitBenchmarker/subscribers","subscription_url":"https://api.github.com/repos/ehankland/PerfKitBenchmarker/subscription","commits_url":"https://api.github.com/repos/ehankland/PerfKitBenchmarker/commits{/sha}","git_commits_url":"https://api.github.com/repos/ehankland/PerfKitBenchmarker/git/commits{/sha}","comments_url":"https://api.github.com/repos/ehankland/PerfKitBenchmarker/comments{/number}","issue_comment_url":"https://api.github.com/repos/ehankland/PerfKitBenchmarker/issues/comments{/number}","contents_url":"https://api.github.com/repos/ehankland/PerfKitBenchmarker/contents/{+path}","compare_url":"https://api.github.com/repos/ehankland/PerfKitBenchmarker/compare/{base}...{head}","merges_url":"https://api.github.com/repos/ehankland/PerfKitBenchmarker/merges","archive_url":"https://api.github.com/repos/ehankland/PerfKitBenchmarker/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/ehankland/PerfKitBenchmarker/downloads","issues_url":"https://api.github.com/repos/ehankland/PerfKitBenchmarker/issues{/number}","pulls_url":"https://api.github.com/repos/ehankland/PerfKitBenchmarker/pulls{/number}","milestones_url":"https://api.github.com/repos/ehankland/PerfKitBenchmarker/milestones{/number}","notifications_url":"https://api.github.com/repos/ehankland/PerfKitBenchmarker/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/ehankland/PerfKitBenchmarker/labels{/name}","releases_url":"https://api.github.com/repos/ehankland/PerfKitBenchmarker/releases{/id}","created_at":"2015-05-27T23:24:42Z","updated_at":"2015-05-27T23:24:43Z","pushed_at":"2015-10-06T17:13:59Z","git_url":"git://github.com/ehankland/PerfKitBenchmarker.git","ssh_url":"git@github.com:ehankland/PerfKitBenchmarker.git","clone_url":"https://github.com/ehankland/PerfKitBenchmarker.git","svn_url":"https://github.com/ehankland/PerfKitBenchmarker","homepage":"","size":3204,"stargazers_count":0,"watchers_count":0,"language":"Python","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"GoogleCloudPlatform:master","ref":"master","sha":"baa8c4b52ccad49e1b68fed6d86e0546183abe28","user":{"login":"GoogleCloudPlatform","id":2810941,"avatar_url":"https://avatars.githubusercontent.com/u/2810941?v=3","gravatar_id":"","url":"https://api.github.com/users/GoogleCloudPlatform","html_url":"https://github.com/GoogleCloudPlatform","followers_url":"https://api.github.com/users/GoogleCloudPlatform/followers","following_url":"https://api.github.com/users/GoogleCloudPlatform/following{/other_user}","gists_url":"https://api.github.com/users/GoogleCloudPlatform/gists{/gist_id}","starred_url":"https://api.github.com/users/GoogleCloudPlatform/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/GoogleCloudPlatform/subscriptions","organizations_url":"https://api.github.com/users/GoogleCloudPlatform/orgs","repos_url":"https://api.github.com/users/GoogleCloudPlatform/repos","events_url":"https://api.github.com/users/GoogleCloudPlatform/events{/privacy}","received_events_url":"https://api.github.com/users/GoogleCloudPlatform/received_events","type":"Organization","site_admin":false},"repo":{"id":21950910,"name":"PerfKitBenchmarker","full_name":"GoogleCloudPlatform/PerfKitBenchmarker","owner":{"login":"GoogleCloudPlatform","id":2810941,"avatar_url":"https://avatars.githubusercontent.com/u/2810941?v=3","gravatar_id":"","url":"https://api.github.com/users/GoogleCloudPlatform","html_url":"https://github.com/GoogleCloudPlatform","followers_url":"https://api.github.com/users/GoogleCloudPlatform/followers","following_url":"https://api.github.com/users/GoogleCloudPlatform/following{/other_user}","gists_url":"https://api.github.com/users/GoogleCloudPlatform/gists{/gist_id}","starred_url":"https://api.github.com/users/GoogleCloudPlatform/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/GoogleCloudPlatform/subscriptions","organizations_url":"https://api.github.com/users/GoogleCloudPlatform/orgs","repos_url":"https://api.github.com/users/GoogleCloudPlatform/repos","events_url":"https://api.github.com/users/GoogleCloudPlatform/events{/privacy}","received_events_url":"https://api.github.com/users/GoogleCloudPlatform/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/GoogleCloudPlatform/PerfKitBenchmarker","description":"Perfkit Benchmarker contains set of benchmarks to measure and compare cloud offerings. The benchmarks are not tuned (i.e. use defaults) to reflect what most users will see. PerfKit Benchmarker is licensed under the Apache 2 license terms. Please make sure to read, understand and agree to the terms of the LICENSE and CONTRIBUTING files before proceeding.","fork":false,"url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker","forks_url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/forks","keys_url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/keys{/key_id}","collaborators_url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/teams","hooks_url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/hooks","issue_events_url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/issues/events{/number}","events_url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/events","assignees_url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/assignees{/user}","branches_url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/branches{/branch}","tags_url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/tags","blobs_url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/git/refs{/sha}","trees_url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/git/trees{/sha}","statuses_url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/statuses/{sha}","languages_url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/languages","stargazers_url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/stargazers","contributors_url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/contributors","subscribers_url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/subscribers","subscription_url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/subscription","commits_url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/commits{/sha}","git_commits_url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/git/commits{/sha}","comments_url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/comments{/number}","issue_comment_url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/issues/comments{/number}","contents_url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/contents/{+path}","compare_url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/compare/{base}...{head}","merges_url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/merges","archive_url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/downloads","issues_url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/issues{/number}","pulls_url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/pulls{/number}","milestones_url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/milestones{/number}","notifications_url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/labels{/name}","releases_url":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/releases{/id}","created_at":"2014-07-17T17:23:26Z","updated_at":"2015-10-05T19:07:17Z","pushed_at":"2015-10-06T17:38:11Z","git_url":"git://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git","ssh_url":"git@github.com:GoogleCloudPlatform/PerfKitBenchmarker.git","clone_url":"https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git","svn_url":"https://github.com/GoogleCloudPlatform/PerfKitBenchmarker","homepage":"","size":85176,"stargazers_count":481,"watchers_count":481,"language":"Python","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":80,"mirror_url":null,"open_issues_count":99,"forks":80,"open_issues":99,"watchers":481,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/pulls/510"},"html":{"href":"https://github.com/GoogleCloudPlatform/PerfKitBenchmarker/pull/510"},"issue":{"href":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/issues/510"},"comments":{"href":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/issues/510/comments"},"review_comments":{"href":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/pulls/510/comments"},"review_comment":{"href":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/pulls/510/commits"},"statuses":{"href":"https://api.github.com/repos/GoogleCloudPlatform/PerfKitBenchmarker/statuses/d6e68dbb7e143ece14b0b5e9e30907ab9cebfa2f"}}}}