{"action":"created","comment":{"url":"https://api.github.com/repos/kubernetes/heapster/pulls/comments/48144169","id":48144169,"diff_hunk":"@@ -0,0 +1,130 @@\n+# Heapster Long Term Vision\n+\n+## Current status\n+\n+Heapster is an important component of Kubernetes that is responsible for metrics and event \n+handling. It reads metrics from cluster nodes and writes them to external, permanent \n+storage. This is the main use case of Heapster.\n+\n+To support system components of Kubernetes Heapster calculates aggregated metrics (like \n+sum of containers' CPU usage in a pod) and long term statistics (average, 95percentile \n+with 1h resolution), keeps them in memory and exposes via Heapster API. This API is mainly\n+used by Horizontal Pod Autoscaler which asks for the most recent performance related \n+metrics to adjust the number of pods to the incoming traffic. The API is also used by KubeDash\n+and will be used by the new UI (which will replace KubeDash) as well. \n+\n+Additionally Heapster API allows to list all active nodes, namespaces, pods, containers \n+etc. present in the system.\n+\n+There is also a HeapsterGKE API dedicated for GKE through which it’s possible to get a full \n+dump of all metrics (spanning last minute or two).\n+\n+Metrics are gathered from cluster nodes, but Heapster developers wanted it to be useful also \n+in non-Kubernetes clusters. They wrote Heapster in a such a way that metrics can be read not \n+only from Kubernetes nodes (via Kubelet API) but also from custom deployments via cAdvisor \n+(with support for CoreOS Fleet and flat file node lists). \n+\n+Metrics collected by Heapster can be written into multiple kinds of storage - Influxdb, \n+OpenTSDB, Google Cloud Monitoring, Hawkular, Kaflka, Riemann, ElasticSearch (some of them are\n+not yet submitted).\n+\n+In addition to gathering metrics  Heapster is responsible for handling Kubernetes events - it \n+reads them from Kubernetes API server and writes them, without extra processing, to a selection\n+of persistent storages: Google Cloud Logging, Influxdb, Kafka, OpenTSDB, Hawkular, \n+ElasticSearch, etc.\n+\n+There is/was a plan to add resource prediction components (Initial Resources, Vertical \n+Pod Autoscaling) to Heapster binary.\n+\n+## Separation of Use Cases\n+From the current state description (see above) the following use cases can be extracted:\n+\n+* [UC1] Read metrics from nodes and write them to an external storage.\n+* [UC2] Expose metrics from the last 2-3 minutes (for HPA and GKE)\n+* [UC3] Read Events from the API server and write them to a permanent storage\n+* [UC4] Do some long-term (hours, days) metrics analysis to get stats (average, 95 percentile) \n+and expected resource usage.\n+* [UC5] Provide cpu and memory metrics for longer time window for the new Kubernetes \n+Dashboard UI (15 min for 1.2, up to 1h later for plots) \n+\n+UC1 and UC2 go together - to expose the most recent metrics the API should be connected \n+to the metrics stream.\n+UC3 can be completely separated from UC1, UC2 and UC4 - it reads different data from a \n+different place and writes it in a slightly different format to different sinks. \n+UC4 is connected to UC1 and UC2 but it is more based on data from the permanent storage \n+than on the super-fresh metrics stored in the memory.\n+UC5 can go either with UC1/UC2 or with UC4. As there is no immediate need for UC4 we will\n+ provide basic UC5 together with UC1/UC2 but in the future it will join UC4.\n+\n+This separation leads to an idea of splitting Heapster into 3 binaries:\n+\n+* Core Heapster - covering UC1, UC2 and temporarily UC5\n+* Eventer - covering UC3\n+* Oldtimer - covering UC4 and UC5 \n+\n+## Reduction of Responsibility\n+\n+With 3 possible node sources (Kuberentes API Server, flat file, CoreOS Fleet), 2 metrics \n+sources (cAdvisor and Kubelet) and constantly growing number of sinks we have to separate \n+the stuff that the core Heapster/K8S team is responsible for and what is provided as a \n+plugin/addition and doesn’t come in the main release package. \n+\n+We decided to focus only on:\n+\n+Kubernetes API Server node source\n+Kubelet metrics source\n+Influxdb, GCM, GKE (there is special endpoint for GKE that exposes all available metrics),\n+ Hawkular sinks for Heapster\n+Influxdb, GCL sinks for Eventer\n+\n+The rest of the sources/sinks will be available as plugins. The plugin will be used in 2 flavors:\n+\n+* Complied in - will require the user to rebuild the package and create his own image with \n+the desired set of plugins.  \n+* Side-car - Heapster will talk to plugin’s HTTP server to get/pass metrics through a well \n+defined json interface. The plugin runs in a separate container.\n+\n+K8s team will explicitly say that it is NOT giving any warranty on the plugins. Plugins e2e \n+tests can be included in some CI suite but we will not block our development (too much) if \n+something breaks Kafka or Riemann. We will also not pay attention to whether a particular sink scales up.\n+\n+For now we will keep all of the currently available sinks compiled-in by default, to keep the \n+new Heapster more or less compatible with the old one, but eventually (if the number of sinks grows)\n+ we will migrate some of them to plugins.\n+\n+## Custom Metrics Status\n+\n+Heapster is not a generic solution for gathering arbitrary number of arbitrary-formated custom \n+metrics. The support for custom metrics is focused on auto-scaling and critical functionality \n+monitoring (and potentially scheduling). And Heapster is oriented towards system metrics, not \n+application/business level metrics.\n+\n+Kubernetes users and application developers will be able to push any number of their custom \n+metrics through our pipeline to the storage but this should be considered as a bonus/best effort \n+functionality. Custom metrics will not influence our performance targets (no extra fine-tunning effort \n+to support >5 custom metrics per pod). There will be a flag in Kubelet that will limit the \n+number of custom metrics.\n+\n+## Performance Target\n+\n+Heapster product family (Core, Eventer and Oldtimer) should follow the same performance goals \n+as core Kubernetes. As Eventer is fairly simple and Oldtimer not yet fully defined this section","path":"docs/proposals/vision.md","position":111,"original_position":111,"commit_id":"88b3105e45b365d663d22728dcac640c9b2c27b3","original_commit_id":"88b3105e45b365d663d22728dcac640c9b2c27b3","user":{"login":"mwielgus","id":11994812,"avatar_url":"https://avatars.githubusercontent.com/u/11994812?v=3","gravatar_id":"","url":"https://api.github.com/users/mwielgus","html_url":"https://github.com/mwielgus","followers_url":"https://api.github.com/users/mwielgus/followers","following_url":"https://api.github.com/users/mwielgus/following{/other_user}","gists_url":"https://api.github.com/users/mwielgus/gists{/gist_id}","starred_url":"https://api.github.com/users/mwielgus/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mwielgus/subscriptions","organizations_url":"https://api.github.com/users/mwielgus/orgs","repos_url":"https://api.github.com/users/mwielgus/repos","events_url":"https://api.github.com/users/mwielgus/events{/privacy}","received_events_url":"https://api.github.com/users/mwielgus/received_events","type":"User","site_admin":false},"body":"There is no official Heapster/Metrics api yet. There is some api for stats but I'm not sure if will be exactly reproduced in Oldtimer. ","created_at":"2015-12-21T13:15:18Z","updated_at":"2015-12-21T13:15:18Z","html_url":"https://github.com/kubernetes/heapster/pull/769#discussion_r48144169","pull_request_url":"https://api.github.com/repos/kubernetes/heapster/pulls/769","_links":{"self":{"href":"https://api.github.com/repos/kubernetes/heapster/pulls/comments/48144169"},"html":{"href":"https://github.com/kubernetes/heapster/pull/769#discussion_r48144169"},"pull_request":{"href":"https://api.github.com/repos/kubernetes/heapster/pulls/769"}}},"pull_request":{"url":"https://api.github.com/repos/kubernetes/heapster/pulls/769","id":52957986,"html_url":"https://github.com/kubernetes/heapster/pull/769","diff_url":"https://github.com/kubernetes/heapster/pull/769.diff","patch_url":"https://github.com/kubernetes/heapster/pull/769.patch","issue_url":"https://api.github.com/repos/kubernetes/heapster/issues/769","number":769,"state":"open","locked":false,"title":"Heapster long term vision","user":{"login":"mwielgus","id":11994812,"avatar_url":"https://avatars.githubusercontent.com/u/11994812?v=3","gravatar_id":"","url":"https://api.github.com/users/mwielgus","html_url":"https://github.com/mwielgus","followers_url":"https://api.github.com/users/mwielgus/followers","following_url":"https://api.github.com/users/mwielgus/following{/other_user}","gists_url":"https://api.github.com/users/mwielgus/gists{/gist_id}","starred_url":"https://api.github.com/users/mwielgus/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mwielgus/subscriptions","organizations_url":"https://api.github.com/users/mwielgus/orgs","repos_url":"https://api.github.com/users/mwielgus/repos","events_url":"https://api.github.com/users/mwielgus/events{/privacy}","received_events_url":"https://api.github.com/users/mwielgus/received_events","type":"User","site_admin":false},"body":"","created_at":"2015-12-08T09:43:25Z","updated_at":"2015-12-21T13:15:18Z","closed_at":null,"merged_at":null,"merge_commit_sha":"543a0291441bd083a862ffc492da4b0e781bb771","assignee":{"login":"fgrzadkowski","id":10820726,"avatar_url":"https://avatars.githubusercontent.com/u/10820726?v=3","gravatar_id":"","url":"https://api.github.com/users/fgrzadkowski","html_url":"https://github.com/fgrzadkowski","followers_url":"https://api.github.com/users/fgrzadkowski/followers","following_url":"https://api.github.com/users/fgrzadkowski/following{/other_user}","gists_url":"https://api.github.com/users/fgrzadkowski/gists{/gist_id}","starred_url":"https://api.github.com/users/fgrzadkowski/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fgrzadkowski/subscriptions","organizations_url":"https://api.github.com/users/fgrzadkowski/orgs","repos_url":"https://api.github.com/users/fgrzadkowski/repos","events_url":"https://api.github.com/users/fgrzadkowski/events{/privacy}","received_events_url":"https://api.github.com/users/fgrzadkowski/received_events","type":"User","site_admin":false},"milestone":null,"commits_url":"https://api.github.com/repos/kubernetes/heapster/pulls/769/commits","review_comments_url":"https://api.github.com/repos/kubernetes/heapster/pulls/769/comments","review_comment_url":"https://api.github.com/repos/kubernetes/heapster/pulls/comments{/number}","comments_url":"https://api.github.com/repos/kubernetes/heapster/issues/769/comments","statuses_url":"https://api.github.com/repos/kubernetes/heapster/statuses/88b3105e45b365d663d22728dcac640c9b2c27b3","head":{"label":"mwielgus:vision","ref":"vision","sha":"88b3105e45b365d663d22728dcac640c9b2c27b3","user":{"login":"mwielgus","id":11994812,"avatar_url":"https://avatars.githubusercontent.com/u/11994812?v=3","gravatar_id":"","url":"https://api.github.com/users/mwielgus","html_url":"https://github.com/mwielgus","followers_url":"https://api.github.com/users/mwielgus/followers","following_url":"https://api.github.com/users/mwielgus/following{/other_user}","gists_url":"https://api.github.com/users/mwielgus/gists{/gist_id}","starred_url":"https://api.github.com/users/mwielgus/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mwielgus/subscriptions","organizations_url":"https://api.github.com/users/mwielgus/orgs","repos_url":"https://api.github.com/users/mwielgus/repos","events_url":"https://api.github.com/users/mwielgus/events{/privacy}","received_events_url":"https://api.github.com/users/mwielgus/received_events","type":"User","site_admin":false},"repo":{"id":40354790,"name":"heapster","full_name":"mwielgus/heapster","owner":{"login":"mwielgus","id":11994812,"avatar_url":"https://avatars.githubusercontent.com/u/11994812?v=3","gravatar_id":"","url":"https://api.github.com/users/mwielgus","html_url":"https://github.com/mwielgus","followers_url":"https://api.github.com/users/mwielgus/followers","following_url":"https://api.github.com/users/mwielgus/following{/other_user}","gists_url":"https://api.github.com/users/mwielgus/gists{/gist_id}","starred_url":"https://api.github.com/users/mwielgus/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mwielgus/subscriptions","organizations_url":"https://api.github.com/users/mwielgus/orgs","repos_url":"https://api.github.com/users/mwielgus/repos","events_url":"https://api.github.com/users/mwielgus/events{/privacy}","received_events_url":"https://api.github.com/users/mwielgus/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/mwielgus/heapster","description":"Monitor container resource usage of a Kubernetes cluster","fork":true,"url":"https://api.github.com/repos/mwielgus/heapster","forks_url":"https://api.github.com/repos/mwielgus/heapster/forks","keys_url":"https://api.github.com/repos/mwielgus/heapster/keys{/key_id}","collaborators_url":"https://api.github.com/repos/mwielgus/heapster/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/mwielgus/heapster/teams","hooks_url":"https://api.github.com/repos/mwielgus/heapster/hooks","issue_events_url":"https://api.github.com/repos/mwielgus/heapster/issues/events{/number}","events_url":"https://api.github.com/repos/mwielgus/heapster/events","assignees_url":"https://api.github.com/repos/mwielgus/heapster/assignees{/user}","branches_url":"https://api.github.com/repos/mwielgus/heapster/branches{/branch}","tags_url":"https://api.github.com/repos/mwielgus/heapster/tags","blobs_url":"https://api.github.com/repos/mwielgus/heapster/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/mwielgus/heapster/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/mwielgus/heapster/git/refs{/sha}","trees_url":"https://api.github.com/repos/mwielgus/heapster/git/trees{/sha}","statuses_url":"https://api.github.com/repos/mwielgus/heapster/statuses/{sha}","languages_url":"https://api.github.com/repos/mwielgus/heapster/languages","stargazers_url":"https://api.github.com/repos/mwielgus/heapster/stargazers","contributors_url":"https://api.github.com/repos/mwielgus/heapster/contributors","subscribers_url":"https://api.github.com/repos/mwielgus/heapster/subscribers","subscription_url":"https://api.github.com/repos/mwielgus/heapster/subscription","commits_url":"https://api.github.com/repos/mwielgus/heapster/commits{/sha}","git_commits_url":"https://api.github.com/repos/mwielgus/heapster/git/commits{/sha}","comments_url":"https://api.github.com/repos/mwielgus/heapster/comments{/number}","issue_comment_url":"https://api.github.com/repos/mwielgus/heapster/issues/comments{/number}","contents_url":"https://api.github.com/repos/mwielgus/heapster/contents/{+path}","compare_url":"https://api.github.com/repos/mwielgus/heapster/compare/{base}...{head}","merges_url":"https://api.github.com/repos/mwielgus/heapster/merges","archive_url":"https://api.github.com/repos/mwielgus/heapster/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/mwielgus/heapster/downloads","issues_url":"https://api.github.com/repos/mwielgus/heapster/issues{/number}","pulls_url":"https://api.github.com/repos/mwielgus/heapster/pulls{/number}","milestones_url":"https://api.github.com/repos/mwielgus/heapster/milestones{/number}","notifications_url":"https://api.github.com/repos/mwielgus/heapster/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/mwielgus/heapster/labels{/name}","releases_url":"https://api.github.com/repos/mwielgus/heapster/releases{/id}","created_at":"2015-08-07T10:18:26Z","updated_at":"2015-08-07T10:18:27Z","pushed_at":"2015-12-15T19:57:15Z","git_url":"git://github.com/mwielgus/heapster.git","ssh_url":"git@github.com:mwielgus/heapster.git","clone_url":"https://github.com/mwielgus/heapster.git","svn_url":"https://github.com/mwielgus/heapster","homepage":"","size":22300,"stargazers_count":0,"watchers_count":0,"language":"Go","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"kubernetes:master","ref":"master","sha":"72400634975c9b30c73a09f98bed84358218405e","user":{"login":"kubernetes","id":13629408,"avatar_url":"https://avatars.githubusercontent.com/u/13629408?v=3","gravatar_id":"","url":"https://api.github.com/users/kubernetes","html_url":"https://github.com/kubernetes","followers_url":"https://api.github.com/users/kubernetes/followers","following_url":"https://api.github.com/users/kubernetes/following{/other_user}","gists_url":"https://api.github.com/users/kubernetes/gists{/gist_id}","starred_url":"https://api.github.com/users/kubernetes/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kubernetes/subscriptions","organizations_url":"https://api.github.com/users/kubernetes/orgs","repos_url":"https://api.github.com/users/kubernetes/repos","events_url":"https://api.github.com/users/kubernetes/events{/privacy}","received_events_url":"https://api.github.com/users/kubernetes/received_events","type":"Organization","site_admin":false},"repo":{"id":23272510,"name":"heapster","full_name":"kubernetes/heapster","owner":{"login":"kubernetes","id":13629408,"avatar_url":"https://avatars.githubusercontent.com/u/13629408?v=3","gravatar_id":"","url":"https://api.github.com/users/kubernetes","html_url":"https://github.com/kubernetes","followers_url":"https://api.github.com/users/kubernetes/followers","following_url":"https://api.github.com/users/kubernetes/following{/other_user}","gists_url":"https://api.github.com/users/kubernetes/gists{/gist_id}","starred_url":"https://api.github.com/users/kubernetes/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kubernetes/subscriptions","organizations_url":"https://api.github.com/users/kubernetes/orgs","repos_url":"https://api.github.com/users/kubernetes/repos","events_url":"https://api.github.com/users/kubernetes/events{/privacy}","received_events_url":"https://api.github.com/users/kubernetes/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/kubernetes/heapster","description":"Compute Resource Usage Analysis and Monitoring of Container Clusters","fork":false,"url":"https://api.github.com/repos/kubernetes/heapster","forks_url":"https://api.github.com/repos/kubernetes/heapster/forks","keys_url":"https://api.github.com/repos/kubernetes/heapster/keys{/key_id}","collaborators_url":"https://api.github.com/repos/kubernetes/heapster/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/kubernetes/heapster/teams","hooks_url":"https://api.github.com/repos/kubernetes/heapster/hooks","issue_events_url":"https://api.github.com/repos/kubernetes/heapster/issues/events{/number}","events_url":"https://api.github.com/repos/kubernetes/heapster/events","assignees_url":"https://api.github.com/repos/kubernetes/heapster/assignees{/user}","branches_url":"https://api.github.com/repos/kubernetes/heapster/branches{/branch}","tags_url":"https://api.github.com/repos/kubernetes/heapster/tags","blobs_url":"https://api.github.com/repos/kubernetes/heapster/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/kubernetes/heapster/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/kubernetes/heapster/git/refs{/sha}","trees_url":"https://api.github.com/repos/kubernetes/heapster/git/trees{/sha}","statuses_url":"https://api.github.com/repos/kubernetes/heapster/statuses/{sha}","languages_url":"https://api.github.com/repos/kubernetes/heapster/languages","stargazers_url":"https://api.github.com/repos/kubernetes/heapster/stargazers","contributors_url":"https://api.github.com/repos/kubernetes/heapster/contributors","subscribers_url":"https://api.github.com/repos/kubernetes/heapster/subscribers","subscription_url":"https://api.github.com/repos/kubernetes/heapster/subscription","commits_url":"https://api.github.com/repos/kubernetes/heapster/commits{/sha}","git_commits_url":"https://api.github.com/repos/kubernetes/heapster/git/commits{/sha}","comments_url":"https://api.github.com/repos/kubernetes/heapster/comments{/number}","issue_comment_url":"https://api.github.com/repos/kubernetes/heapster/issues/comments{/number}","contents_url":"https://api.github.com/repos/kubernetes/heapster/contents/{+path}","compare_url":"https://api.github.com/repos/kubernetes/heapster/compare/{base}...{head}","merges_url":"https://api.github.com/repos/kubernetes/heapster/merges","archive_url":"https://api.github.com/repos/kubernetes/heapster/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/kubernetes/heapster/downloads","issues_url":"https://api.github.com/repos/kubernetes/heapster/issues{/number}","pulls_url":"https://api.github.com/repos/kubernetes/heapster/pulls{/number}","milestones_url":"https://api.github.com/repos/kubernetes/heapster/milestones{/number}","notifications_url":"https://api.github.com/repos/kubernetes/heapster/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/kubernetes/heapster/labels{/name}","releases_url":"https://api.github.com/repos/kubernetes/heapster/releases{/id}","created_at":"2014-08-24T03:50:18Z","updated_at":"2015-12-21T11:17:31Z","pushed_at":"2015-12-18T22:08:06Z","git_url":"git://github.com/kubernetes/heapster.git","ssh_url":"git@github.com:kubernetes/heapster.git","clone_url":"https://github.com/kubernetes/heapster.git","svn_url":"https://github.com/kubernetes/heapster","homepage":"","size":23358,"stargazers_count":514,"watchers_count":514,"language":"Go","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":148,"mirror_url":null,"open_issues_count":142,"forks":148,"open_issues":142,"watchers":514,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/kubernetes/heapster/pulls/769"},"html":{"href":"https://github.com/kubernetes/heapster/pull/769"},"issue":{"href":"https://api.github.com/repos/kubernetes/heapster/issues/769"},"comments":{"href":"https://api.github.com/repos/kubernetes/heapster/issues/769/comments"},"review_comments":{"href":"https://api.github.com/repos/kubernetes/heapster/pulls/769/comments"},"review_comment":{"href":"https://api.github.com/repos/kubernetes/heapster/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/kubernetes/heapster/pulls/769/commits"},"statuses":{"href":"https://api.github.com/repos/kubernetes/heapster/statuses/88b3105e45b365d663d22728dcac640c9b2c27b3"}}}}