{"action":"created","comment":{"url":"https://api.github.com/repos/kubernetes/kubernetes/pulls/comments/41343323","id":41343323,"diff_hunk":"@@ -0,0 +1,302 @@\n+<!-- BEGIN MUNGE: UNVERSIONED_WARNING -->\n+\n+<!-- BEGIN STRIP_FOR_RELEASE -->\n+\n+<img src=\"http://kubernetes.io/img/warning.png\" alt=\"WARNING\"\n+     width=\"25\" height=\"25\">\n+<img src=\"http://kubernetes.io/img/warning.png\" alt=\"WARNING\"\n+     width=\"25\" height=\"25\">\n+<img src=\"http://kubernetes.io/img/warning.png\" alt=\"WARNING\"\n+     width=\"25\" height=\"25\">\n+<img src=\"http://kubernetes.io/img/warning.png\" alt=\"WARNING\"\n+     width=\"25\" height=\"25\">\n+<img src=\"http://kubernetes.io/img/warning.png\" alt=\"WARNING\"\n+     width=\"25\" height=\"25\">\n+\n+<h2>PLEASE NOTE: This document applies to the HEAD of the source tree</h2>\n+\n+If you are using a released version of Kubernetes, you should\n+refer to the docs that go with that version.\n+\n+<strong>\n+The latest 1.0.x release of this document can be found\n+[here](http://releases.k8s.io/release-1.0/docs/design/aws_under_the_hood.md).\n+\n+Documentation for other releases can be found at\n+[releases.k8s.io](http://releases.k8s.io).\n+</strong>\n+--\n+\n+<!-- END STRIP_FOR_RELEASE -->\n+\n+<!-- END MUNGE: UNVERSIONED_WARNING -->\n+\n+# Peeking under the hood of Kubernetes on AWS\n+\n+This document provides high-level insight into how Kubernetes works on AWS and\n+maps to AWS objects.  We assume that you are familiar with AWS.\n+\n+We encourage you to use [kube-up](../getting-started-guides/aws.md) (or\n+[CloudFormation](../getting-started-guides/aws-coreos.md)) to create clusters on\n+AWS. We recommend that you avoid manual configuration but are aware that\n+sometimes it's the only option.\n+\n+Tip: You should open an issue and let us know what enhancements can be made to\n+the scripts to better suit your needs.\n+\n+That said, it's also useful to know what's happening under the hood when\n+Kubernetes clusters are created on AWS. This can be particularly useful if\n+problems arise or in circumstances where the provided scripts are lacking and\n+you manually created or configured your cluster.\n+\n+### Architecture overview\n+\n+Kubernetes is a cluster of several machines that consists of a Kubernetes\n+master and a set number of nodes (previously known as 'minions') for which the\n+master which is responsible. See the [Architecture](architecture.md) topic for\n+more details.\n+\n+Other documents describe the general architecture of Kubernetes (all nodes run\n+Docker; the kubelet agent runs on each node and launches containers; the\n+kube-proxy relays traffic between the nodes etc).\n+\n+By default on AWS:\n+\n+* Instances run Ubuntu 15.04 (the official AMI).  It includes a sufficiently\n+  modern kernel that pairs well with Docker and doesn't require a\n+  reboot.  (The default SSH user is `ubuntu` for this and other ubuntu images.)\n+* By default we run aufs over ext4 as the filesystem / container storage on the\n+  nodes (mostly because this is what GCE uses).\n+\n+You can override these defaults by passing different environment variables to\n+kube-up.\n+\n+### Storage\n+\n+AWS supports persistent volumes by using [Elastic Block Store (EBS)](../user-guide/volumes.md#awselasticblockstore).  These can then be\n+attached to pods that should store persistent data (e.g. if you're running a\n+database).\n+\n+By default, nodes in AWS use [instance storage](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/InstanceStorage.html)\n+unless you create pods with persistent volumes\n+[(EBS)](../user-guide/volumes.md#awselasticblockstore).  In general, Kubernetes\n+containers do not have persistent storage unless you attach a persistent\n+volume, and so nodes on AWS use instance storage.  Instance storage is cheaper,\n+often faster, and historically more reliable.  This does mean that you should\n+pick an instance type that has sufficient instance storage, unless you can make\n+do with whatever space is left on your root partition.\n+\n+Note: The master uses a persistent volume ([etcd](architecture.md#etcd)) to track\n+its state but similar to the nodes, containers are mostly run against instance\n+storage, except that we repoint some important data onto the peristent volume.\n+\n+The default storage driver for Docker images is aufs.  Specifying btrfs (by passing the environment\n+variable `DOCKER_STORAGE=btrfs` to kube-up) is also a good choice for a filesystem.  btrfs\n+is relatively reliable with Docker and has improved its reliability with modern\n+kernels.  It can easily span multiple volumes, which is particularly useful\n+when we are using an instance type with multiple ephemeral instance disks.\n+\n+### AutoScaling\n+\n+Nodes (but not the master) are run in an\n+[AutoScalingGroup](http://docs.aws.amazon.com/AutoScaling/latest/DeveloperGuide/AutoScalingGroup.html)\n+on AWS.  Currently auto-scaling (e.g. based on CPU) is not actually enabled\n+([#11935](http://issues.k8s.io/11935)).  Instead, the auto-scaling group means\n+that AWS will relaunch any nodes that are terminated.\n+\n+We do not currently run the master in an AutoScalingGroup, but we should\n+([#11934](http://issues.k8s.io/11934)).\n+\n+### Networking\n+\n+Kubernetes uses an IP-per-pod model.  This means that a node, which runs many\n+pods, must have many IPs.  AWS uses virtual private clouds (VPCs) and advanced\n+routing support so each pod is assigned a /24 CIDR.  Each pod is assigned a /24\n+CIDR; the assigned CIDR is then configured to route to an instance in the VPC\n+routing table.\n+\n+It is also possible to use overlay networking on AWS, but that is not the","path":"docs/design/aws_under_the_hood.md","position":118,"original_position":118,"commit_id":"c7b2fdabb93cab1e223da8b3a269a02c0a7315fa","original_commit_id":"c7b2fdabb93cab1e223da8b3a269a02c0a7315fa","user":{"login":"RichieEscarez","id":11762685,"avatar_url":"https://avatars.githubusercontent.com/u/11762685?v=3","gravatar_id":"","url":"https://api.github.com/users/RichieEscarez","html_url":"https://github.com/RichieEscarez","followers_url":"https://api.github.com/users/RichieEscarez/followers","following_url":"https://api.github.com/users/RichieEscarez/following{/other_user}","gists_url":"https://api.github.com/users/RichieEscarez/gists{/gist_id}","starred_url":"https://api.github.com/users/RichieEscarez/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/RichieEscarez/subscriptions","organizations_url":"https://api.github.com/users/RichieEscarez/orgs","repos_url":"https://api.github.com/users/RichieEscarez/repos","events_url":"https://api.github.com/users/RichieEscarez/events{/privacy}","received_events_url":"https://api.github.com/users/RichieEscarez/received_events","type":"User","site_admin":false},"body":"Insert \"default\":\r\n\r\n\"....but that is not the default configuration of the...\"","created_at":"2015-10-07T01:31:27Z","updated_at":"2015-10-07T01:31:27Z","html_url":"https://github.com/kubernetes/kubernetes/pull/11938#discussion_r41343323","pull_request_url":"https://api.github.com/repos/kubernetes/kubernetes/pulls/11938","_links":{"self":{"href":"https://api.github.com/repos/kubernetes/kubernetes/pulls/comments/41343323"},"html":{"href":"https://github.com/kubernetes/kubernetes/pull/11938#discussion_r41343323"},"pull_request":{"href":"https://api.github.com/repos/kubernetes/kubernetes/pulls/11938"}}},"pull_request":{"url":"https://api.github.com/repos/kubernetes/kubernetes/pulls/11938","id":41043588,"html_url":"https://github.com/kubernetes/kubernetes/pull/11938","diff_url":"https://github.com/kubernetes/kubernetes/pull/11938.diff","patch_url":"https://github.com/kubernetes/kubernetes/pull/11938.patch","issue_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/11938","number":11938,"state":"open","locked":false,"title":"AWS 'under the hood' document","user":{"login":"justinsb","id":100893,"avatar_url":"https://avatars.githubusercontent.com/u/100893?v=3","gravatar_id":"","url":"https://api.github.com/users/justinsb","html_url":"https://github.com/justinsb","followers_url":"https://api.github.com/users/justinsb/followers","following_url":"https://api.github.com/users/justinsb/following{/other_user}","gists_url":"https://api.github.com/users/justinsb/gists{/gist_id}","starred_url":"https://api.github.com/users/justinsb/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/justinsb/subscriptions","organizations_url":"https://api.github.com/users/justinsb/orgs","repos_url":"https://api.github.com/users/justinsb/repos","events_url":"https://api.github.com/users/justinsb/events{/privacy}","received_events_url":"https://api.github.com/users/justinsb/received_events","type":"User","site_admin":false},"body":"Edit: I don't think WIP any longer, but comments still very welcome (and this is definitely v1).\r\n\r\nThis is a WIP PR to allow people to comment - comments are very welcome!\r\n\r\nA lot of people have asked for a \"what is kube-up\" doing document, which describes the internals of what kube-up creates.   The intention is that this document will be sufficiently detailed that somebody could write a replacement for kube-up, or bring up kubernetes on AWS manually (with a lot of work, admittedly), or make sure that our CloudFormation templates are doing the same thing.\r\n\r\nThis is my first draft, very WIP document, mostly so I can provide a place for people to provide feedback.  Please feel free to add lots of questions and requests for more detail.\r\n\r\n(This has also proved very helpful to me in identifying some of the remaining functionality gaps, which is why I just opened that series of AWS issues!)","created_at":"2015-07-28T18:24:10Z","updated_at":"2015-10-07T01:31:27Z","closed_at":null,"merged_at":null,"merge_commit_sha":"19994c202f5c44e6de46cb51ac70c2b25160cde6","assignee":{"login":"RichieEscarez","id":11762685,"avatar_url":"https://avatars.githubusercontent.com/u/11762685?v=3","gravatar_id":"","url":"https://api.github.com/users/RichieEscarez","html_url":"https://github.com/RichieEscarez","followers_url":"https://api.github.com/users/RichieEscarez/followers","following_url":"https://api.github.com/users/RichieEscarez/following{/other_user}","gists_url":"https://api.github.com/users/RichieEscarez/gists{/gist_id}","starred_url":"https://api.github.com/users/RichieEscarez/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/RichieEscarez/subscriptions","organizations_url":"https://api.github.com/users/RichieEscarez/orgs","repos_url":"https://api.github.com/users/RichieEscarez/repos","events_url":"https://api.github.com/users/RichieEscarez/events{/privacy}","received_events_url":"https://api.github.com/users/RichieEscarez/received_events","type":"User","site_admin":false},"milestone":null,"commits_url":"https://api.github.com/repos/kubernetes/kubernetes/pulls/11938/commits","review_comments_url":"https://api.github.com/repos/kubernetes/kubernetes/pulls/11938/comments","review_comment_url":"https://api.github.com/repos/kubernetes/kubernetes/pulls/comments{/number}","comments_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/11938/comments","statuses_url":"https://api.github.com/repos/kubernetes/kubernetes/statuses/c7b2fdabb93cab1e223da8b3a269a02c0a7315fa","head":{"label":"justinsb:docs_aws_internals","ref":"docs_aws_internals","sha":"c7b2fdabb93cab1e223da8b3a269a02c0a7315fa","user":{"login":"justinsb","id":100893,"avatar_url":"https://avatars.githubusercontent.com/u/100893?v=3","gravatar_id":"","url":"https://api.github.com/users/justinsb","html_url":"https://github.com/justinsb","followers_url":"https://api.github.com/users/justinsb/followers","following_url":"https://api.github.com/users/justinsb/following{/other_user}","gists_url":"https://api.github.com/users/justinsb/gists{/gist_id}","starred_url":"https://api.github.com/users/justinsb/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/justinsb/subscriptions","organizations_url":"https://api.github.com/users/justinsb/orgs","repos_url":"https://api.github.com/users/justinsb/repos","events_url":"https://api.github.com/users/justinsb/events{/privacy}","received_events_url":"https://api.github.com/users/justinsb/received_events","type":"User","site_admin":false},"repo":{"id":26103010,"name":"kubernetes","full_name":"justinsb/kubernetes","owner":{"login":"justinsb","id":100893,"avatar_url":"https://avatars.githubusercontent.com/u/100893?v=3","gravatar_id":"","url":"https://api.github.com/users/justinsb","html_url":"https://github.com/justinsb","followers_url":"https://api.github.com/users/justinsb/followers","following_url":"https://api.github.com/users/justinsb/following{/other_user}","gists_url":"https://api.github.com/users/justinsb/gists{/gist_id}","starred_url":"https://api.github.com/users/justinsb/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/justinsb/subscriptions","organizations_url":"https://api.github.com/users/justinsb/orgs","repos_url":"https://api.github.com/users/justinsb/repos","events_url":"https://api.github.com/users/justinsb/events{/privacy}","received_events_url":"https://api.github.com/users/justinsb/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/justinsb/kubernetes","description":"Container Cluster Manager","fork":true,"url":"https://api.github.com/repos/justinsb/kubernetes","forks_url":"https://api.github.com/repos/justinsb/kubernetes/forks","keys_url":"https://api.github.com/repos/justinsb/kubernetes/keys{/key_id}","collaborators_url":"https://api.github.com/repos/justinsb/kubernetes/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/justinsb/kubernetes/teams","hooks_url":"https://api.github.com/repos/justinsb/kubernetes/hooks","issue_events_url":"https://api.github.com/repos/justinsb/kubernetes/issues/events{/number}","events_url":"https://api.github.com/repos/justinsb/kubernetes/events","assignees_url":"https://api.github.com/repos/justinsb/kubernetes/assignees{/user}","branches_url":"https://api.github.com/repos/justinsb/kubernetes/branches{/branch}","tags_url":"https://api.github.com/repos/justinsb/kubernetes/tags","blobs_url":"https://api.github.com/repos/justinsb/kubernetes/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/justinsb/kubernetes/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/justinsb/kubernetes/git/refs{/sha}","trees_url":"https://api.github.com/repos/justinsb/kubernetes/git/trees{/sha}","statuses_url":"https://api.github.com/repos/justinsb/kubernetes/statuses/{sha}","languages_url":"https://api.github.com/repos/justinsb/kubernetes/languages","stargazers_url":"https://api.github.com/repos/justinsb/kubernetes/stargazers","contributors_url":"https://api.github.com/repos/justinsb/kubernetes/contributors","subscribers_url":"https://api.github.com/repos/justinsb/kubernetes/subscribers","subscription_url":"https://api.github.com/repos/justinsb/kubernetes/subscription","commits_url":"https://api.github.com/repos/justinsb/kubernetes/commits{/sha}","git_commits_url":"https://api.github.com/repos/justinsb/kubernetes/git/commits{/sha}","comments_url":"https://api.github.com/repos/justinsb/kubernetes/comments{/number}","issue_comment_url":"https://api.github.com/repos/justinsb/kubernetes/issues/comments{/number}","contents_url":"https://api.github.com/repos/justinsb/kubernetes/contents/{+path}","compare_url":"https://api.github.com/repos/justinsb/kubernetes/compare/{base}...{head}","merges_url":"https://api.github.com/repos/justinsb/kubernetes/merges","archive_url":"https://api.github.com/repos/justinsb/kubernetes/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/justinsb/kubernetes/downloads","issues_url":"https://api.github.com/repos/justinsb/kubernetes/issues{/number}","pulls_url":"https://api.github.com/repos/justinsb/kubernetes/pulls{/number}","milestones_url":"https://api.github.com/repos/justinsb/kubernetes/milestones{/number}","notifications_url":"https://api.github.com/repos/justinsb/kubernetes/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/justinsb/kubernetes/labels{/name}","releases_url":"https://api.github.com/repos/justinsb/kubernetes/releases{/id}","created_at":"2014-11-03T03:53:05Z","updated_at":"2015-02-24T20:55:05Z","pushed_at":"2015-10-06T22:07:32Z","git_url":"git://github.com/justinsb/kubernetes.git","ssh_url":"git@github.com:justinsb/kubernetes.git","clone_url":"https://github.com/justinsb/kubernetes.git","svn_url":"https://github.com/justinsb/kubernetes","homepage":null,"size":115488,"stargazers_count":1,"watchers_count":1,"language":"Go","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":true,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":1,"default_branch":"master"}},"base":{"label":"kubernetes:master","ref":"master","sha":"9d1a48d5f694926581a98a4b4315977bb386e5aa","user":{"login":"kubernetes","id":13629408,"avatar_url":"https://avatars.githubusercontent.com/u/13629408?v=3","gravatar_id":"","url":"https://api.github.com/users/kubernetes","html_url":"https://github.com/kubernetes","followers_url":"https://api.github.com/users/kubernetes/followers","following_url":"https://api.github.com/users/kubernetes/following{/other_user}","gists_url":"https://api.github.com/users/kubernetes/gists{/gist_id}","starred_url":"https://api.github.com/users/kubernetes/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kubernetes/subscriptions","organizations_url":"https://api.github.com/users/kubernetes/orgs","repos_url":"https://api.github.com/users/kubernetes/repos","events_url":"https://api.github.com/users/kubernetes/events{/privacy}","received_events_url":"https://api.github.com/users/kubernetes/received_events","type":"Organization","site_admin":false},"repo":{"id":20580498,"name":"kubernetes","full_name":"kubernetes/kubernetes","owner":{"login":"kubernetes","id":13629408,"avatar_url":"https://avatars.githubusercontent.com/u/13629408?v=3","gravatar_id":"","url":"https://api.github.com/users/kubernetes","html_url":"https://github.com/kubernetes","followers_url":"https://api.github.com/users/kubernetes/followers","following_url":"https://api.github.com/users/kubernetes/following{/other_user}","gists_url":"https://api.github.com/users/kubernetes/gists{/gist_id}","starred_url":"https://api.github.com/users/kubernetes/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kubernetes/subscriptions","organizations_url":"https://api.github.com/users/kubernetes/orgs","repos_url":"https://api.github.com/users/kubernetes/repos","events_url":"https://api.github.com/users/kubernetes/events{/privacy}","received_events_url":"https://api.github.com/users/kubernetes/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/kubernetes/kubernetes","description":"Container Cluster Manager from Google","fork":false,"url":"https://api.github.com/repos/kubernetes/kubernetes","forks_url":"https://api.github.com/repos/kubernetes/kubernetes/forks","keys_url":"https://api.github.com/repos/kubernetes/kubernetes/keys{/key_id}","collaborators_url":"https://api.github.com/repos/kubernetes/kubernetes/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/kubernetes/kubernetes/teams","hooks_url":"https://api.github.com/repos/kubernetes/kubernetes/hooks","issue_events_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/events{/number}","events_url":"https://api.github.com/repos/kubernetes/kubernetes/events","assignees_url":"https://api.github.com/repos/kubernetes/kubernetes/assignees{/user}","branches_url":"https://api.github.com/repos/kubernetes/kubernetes/branches{/branch}","tags_url":"https://api.github.com/repos/kubernetes/kubernetes/tags","blobs_url":"https://api.github.com/repos/kubernetes/kubernetes/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/kubernetes/kubernetes/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/kubernetes/kubernetes/git/refs{/sha}","trees_url":"https://api.github.com/repos/kubernetes/kubernetes/git/trees{/sha}","statuses_url":"https://api.github.com/repos/kubernetes/kubernetes/statuses/{sha}","languages_url":"https://api.github.com/repos/kubernetes/kubernetes/languages","stargazers_url":"https://api.github.com/repos/kubernetes/kubernetes/stargazers","contributors_url":"https://api.github.com/repos/kubernetes/kubernetes/contributors","subscribers_url":"https://api.github.com/repos/kubernetes/kubernetes/subscribers","subscription_url":"https://api.github.com/repos/kubernetes/kubernetes/subscription","commits_url":"https://api.github.com/repos/kubernetes/kubernetes/commits{/sha}","git_commits_url":"https://api.github.com/repos/kubernetes/kubernetes/git/commits{/sha}","comments_url":"https://api.github.com/repos/kubernetes/kubernetes/comments{/number}","issue_comment_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/comments{/number}","contents_url":"https://api.github.com/repos/kubernetes/kubernetes/contents/{+path}","compare_url":"https://api.github.com/repos/kubernetes/kubernetes/compare/{base}...{head}","merges_url":"https://api.github.com/repos/kubernetes/kubernetes/merges","archive_url":"https://api.github.com/repos/kubernetes/kubernetes/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/kubernetes/kubernetes/downloads","issues_url":"https://api.github.com/repos/kubernetes/kubernetes/issues{/number}","pulls_url":"https://api.github.com/repos/kubernetes/kubernetes/pulls{/number}","milestones_url":"https://api.github.com/repos/kubernetes/kubernetes/milestones{/number}","notifications_url":"https://api.github.com/repos/kubernetes/kubernetes/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/kubernetes/kubernetes/labels{/name}","releases_url":"https://api.github.com/repos/kubernetes/kubernetes/releases{/id}","created_at":"2014-06-06T22:56:04Z","updated_at":"2015-10-06T23:46:25Z","pushed_at":"2015-10-07T01:30:56Z","git_url":"git://github.com/kubernetes/kubernetes.git","ssh_url":"git@github.com:kubernetes/kubernetes.git","clone_url":"https://github.com/kubernetes/kubernetes.git","svn_url":"https://github.com/kubernetes/kubernetes","homepage":"http://kubernetes.io","size":440683,"stargazers_count":10582,"watchers_count":10582,"language":"Go","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":true,"forks_count":2658,"mirror_url":null,"open_issues_count":2007,"forks":2658,"open_issues":2007,"watchers":10582,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/kubernetes/kubernetes/pulls/11938"},"html":{"href":"https://github.com/kubernetes/kubernetes/pull/11938"},"issue":{"href":"https://api.github.com/repos/kubernetes/kubernetes/issues/11938"},"comments":{"href":"https://api.github.com/repos/kubernetes/kubernetes/issues/11938/comments"},"review_comments":{"href":"https://api.github.com/repos/kubernetes/kubernetes/pulls/11938/comments"},"review_comment":{"href":"https://api.github.com/repos/kubernetes/kubernetes/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/kubernetes/kubernetes/pulls/11938/commits"},"statuses":{"href":"https://api.github.com/repos/kubernetes/kubernetes/statuses/c7b2fdabb93cab1e223da8b3a269a02c0a7315fa"}}}}