{"action":"created","comment":{"url":"https://api.github.com/repos/kubernetes/heapster/pulls/comments/47012314","id":47012314,"diff_hunk":"@@ -0,0 +1,131 @@\n+# Heapster Long Term Vision\n+\n+## Current status\n+\n+Heapster is an important component of Kubernetes that is responsible for metrics and event \n+handling. It reads metrics from cluster nodes and writes them to external, permanent \n+storage. This is the main use case of Heapster.\n+\n+To support system components of Kubernetes Heapster calculates aggregated metrics (like \n+sum of containers' CPU usage in a pod) and long term statistics (average, 95percentile \n+with 1h resolution), keeps them in memory and exposes via Heapster API. This API is mainly\n+used by Horizontal Pod Autoscaler which asks for the most recent performance related \n+metrics to adjust the number of pods to the incoming traffic. The API is also used by KubeDash\n+(which unfortunately didn’t get enough traction and will be replaced) and will be used \n+by the new UI as well. \n+\n+Additionally Heapster API allows to list all active nodes, namespaces, pods, containers \n+etc. present in the system.\n+\n+There is also a HeapsterGKE API dedicated for GKE through which it’s possible to get a full \n+dump of all metrics (spanning last minute or two).\n+\n+Metrics are gathered from cluster nodes, but Heapster developers wanted it to be useful also \n+in non-Kubernetes clusters. They wrote Heapster in a such a way that metrics can be read not \n+only from Kubernetes nodes (via Kubelet API) but also from custom deployments via cAdvisor \n+(with support for CoreOS Fleet and flat file node lists). \n+\n+Metrics collected by Heapster can be written into multiple kinds of storage - Influxdb, \n+OpenTSDB, Google Cloud Monitoring, Hawkular, Kaflka, Riemann, ElasticSearch (some of them are\n+not yet submitted).\n+\n+In addition to gathering metrics  Heapster is responsible for handling Kubernetes events - it \n+reads them from Kubernetes API server and writes them, without extra processing, to a selection\n+of persistent storages: Google Cloud Logging, Influxdb, Kafka, OpenTSDB, Hawkular, \n+ElasticSearch, etc.\n+\n+There is/was a plan to add resource prediction components (Initial Resources, Vertical \n+Pod Autoscaling) to Heapster binary.\n+\n+## Separation of Use Cases\n+From the current state description (see above) the following use cases can be extracted:\n+\n+* [UC1] Read metrics from nodes and write them to an external storage.\n+* [UC2] Expose metrics from the last 2-3 minutes (for HPA and GKE)\n+* [UC3] Read Events from the API server and write them to a permanent storage\n+* [UC4] Do some long-term (hours, days) metrics analysis to get stats (average, 95 percentile) \n+and expected resource usage.\n+* [UC5] Provide cpu and memory metrics for longer time window for the new Kubernetes ","path":"docs/proposals/vision.md","position":48,"original_position":48,"commit_id":"5e14d43c173c1135745248ba60fe8eb0b72003b8","original_commit_id":"5e14d43c173c1135745248ba60fe8eb0b72003b8","user":{"login":"mwielgus","id":11994812,"avatar_url":"https://avatars.githubusercontent.com/u/11994812?v=3","gravatar_id":"","url":"https://api.github.com/users/mwielgus","html_url":"https://github.com/mwielgus","followers_url":"https://api.github.com/users/mwielgus/followers","following_url":"https://api.github.com/users/mwielgus/following{/other_user}","gists_url":"https://api.github.com/users/mwielgus/gists{/gist_id}","starred_url":"https://api.github.com/users/mwielgus/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mwielgus/subscriptions","organizations_url":"https://api.github.com/users/mwielgus/orgs","repos_url":"https://api.github.com/users/mwielgus/repos","events_url":"https://api.github.com/users/mwielgus/events{/privacy}","received_events_url":"https://api.github.com/users/mwielgus/received_events","type":"User","site_admin":false},"body":"Well this is a compromise. Storing all of the data for 15 min will consume lots of memory for nothing (no use case). UI and some Vish's experimental scheduler needs cpu and mem info for longer time window (preferably 1h). ","created_at":"2015-12-08T20:56:20Z","updated_at":"2015-12-08T20:56:20Z","html_url":"https://github.com/kubernetes/heapster/pull/769#discussion_r47012314","pull_request_url":"https://api.github.com/repos/kubernetes/heapster/pulls/769","_links":{"self":{"href":"https://api.github.com/repos/kubernetes/heapster/pulls/comments/47012314"},"html":{"href":"https://github.com/kubernetes/heapster/pull/769#discussion_r47012314"},"pull_request":{"href":"https://api.github.com/repos/kubernetes/heapster/pulls/769"}}},"pull_request":{"url":"https://api.github.com/repos/kubernetes/heapster/pulls/769","id":52957986,"html_url":"https://github.com/kubernetes/heapster/pull/769","diff_url":"https://github.com/kubernetes/heapster/pull/769.diff","patch_url":"https://github.com/kubernetes/heapster/pull/769.patch","issue_url":"https://api.github.com/repos/kubernetes/heapster/issues/769","number":769,"state":"open","locked":false,"title":"Heapster long term vision","user":{"login":"mwielgus","id":11994812,"avatar_url":"https://avatars.githubusercontent.com/u/11994812?v=3","gravatar_id":"","url":"https://api.github.com/users/mwielgus","html_url":"https://github.com/mwielgus","followers_url":"https://api.github.com/users/mwielgus/followers","following_url":"https://api.github.com/users/mwielgus/following{/other_user}","gists_url":"https://api.github.com/users/mwielgus/gists{/gist_id}","starred_url":"https://api.github.com/users/mwielgus/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mwielgus/subscriptions","organizations_url":"https://api.github.com/users/mwielgus/orgs","repos_url":"https://api.github.com/users/mwielgus/repos","events_url":"https://api.github.com/users/mwielgus/events{/privacy}","received_events_url":"https://api.github.com/users/mwielgus/received_events","type":"User","site_admin":false},"body":"","created_at":"2015-12-08T09:43:25Z","updated_at":"2015-12-08T20:56:20Z","closed_at":null,"merged_at":null,"merge_commit_sha":"bac5cd162a5ca8d0dbd487254fd7cd0fd26c979d","assignee":{"login":"fgrzadkowski","id":10820726,"avatar_url":"https://avatars.githubusercontent.com/u/10820726?v=3","gravatar_id":"","url":"https://api.github.com/users/fgrzadkowski","html_url":"https://github.com/fgrzadkowski","followers_url":"https://api.github.com/users/fgrzadkowski/followers","following_url":"https://api.github.com/users/fgrzadkowski/following{/other_user}","gists_url":"https://api.github.com/users/fgrzadkowski/gists{/gist_id}","starred_url":"https://api.github.com/users/fgrzadkowski/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fgrzadkowski/subscriptions","organizations_url":"https://api.github.com/users/fgrzadkowski/orgs","repos_url":"https://api.github.com/users/fgrzadkowski/repos","events_url":"https://api.github.com/users/fgrzadkowski/events{/privacy}","received_events_url":"https://api.github.com/users/fgrzadkowski/received_events","type":"User","site_admin":false},"milestone":null,"commits_url":"https://api.github.com/repos/kubernetes/heapster/pulls/769/commits","review_comments_url":"https://api.github.com/repos/kubernetes/heapster/pulls/769/comments","review_comment_url":"https://api.github.com/repos/kubernetes/heapster/pulls/comments{/number}","comments_url":"https://api.github.com/repos/kubernetes/heapster/issues/769/comments","statuses_url":"https://api.github.com/repos/kubernetes/heapster/statuses/5e14d43c173c1135745248ba60fe8eb0b72003b8","head":{"label":"mwielgus:vision","ref":"vision","sha":"5e14d43c173c1135745248ba60fe8eb0b72003b8","user":{"login":"mwielgus","id":11994812,"avatar_url":"https://avatars.githubusercontent.com/u/11994812?v=3","gravatar_id":"","url":"https://api.github.com/users/mwielgus","html_url":"https://github.com/mwielgus","followers_url":"https://api.github.com/users/mwielgus/followers","following_url":"https://api.github.com/users/mwielgus/following{/other_user}","gists_url":"https://api.github.com/users/mwielgus/gists{/gist_id}","starred_url":"https://api.github.com/users/mwielgus/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mwielgus/subscriptions","organizations_url":"https://api.github.com/users/mwielgus/orgs","repos_url":"https://api.github.com/users/mwielgus/repos","events_url":"https://api.github.com/users/mwielgus/events{/privacy}","received_events_url":"https://api.github.com/users/mwielgus/received_events","type":"User","site_admin":false},"repo":{"id":40354790,"name":"heapster","full_name":"mwielgus/heapster","owner":{"login":"mwielgus","id":11994812,"avatar_url":"https://avatars.githubusercontent.com/u/11994812?v=3","gravatar_id":"","url":"https://api.github.com/users/mwielgus","html_url":"https://github.com/mwielgus","followers_url":"https://api.github.com/users/mwielgus/followers","following_url":"https://api.github.com/users/mwielgus/following{/other_user}","gists_url":"https://api.github.com/users/mwielgus/gists{/gist_id}","starred_url":"https://api.github.com/users/mwielgus/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mwielgus/subscriptions","organizations_url":"https://api.github.com/users/mwielgus/orgs","repos_url":"https://api.github.com/users/mwielgus/repos","events_url":"https://api.github.com/users/mwielgus/events{/privacy}","received_events_url":"https://api.github.com/users/mwielgus/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/mwielgus/heapster","description":"Monitor container resource usage of a Kubernetes cluster","fork":true,"url":"https://api.github.com/repos/mwielgus/heapster","forks_url":"https://api.github.com/repos/mwielgus/heapster/forks","keys_url":"https://api.github.com/repos/mwielgus/heapster/keys{/key_id}","collaborators_url":"https://api.github.com/repos/mwielgus/heapster/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/mwielgus/heapster/teams","hooks_url":"https://api.github.com/repos/mwielgus/heapster/hooks","issue_events_url":"https://api.github.com/repos/mwielgus/heapster/issues/events{/number}","events_url":"https://api.github.com/repos/mwielgus/heapster/events","assignees_url":"https://api.github.com/repos/mwielgus/heapster/assignees{/user}","branches_url":"https://api.github.com/repos/mwielgus/heapster/branches{/branch}","tags_url":"https://api.github.com/repos/mwielgus/heapster/tags","blobs_url":"https://api.github.com/repos/mwielgus/heapster/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/mwielgus/heapster/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/mwielgus/heapster/git/refs{/sha}","trees_url":"https://api.github.com/repos/mwielgus/heapster/git/trees{/sha}","statuses_url":"https://api.github.com/repos/mwielgus/heapster/statuses/{sha}","languages_url":"https://api.github.com/repos/mwielgus/heapster/languages","stargazers_url":"https://api.github.com/repos/mwielgus/heapster/stargazers","contributors_url":"https://api.github.com/repos/mwielgus/heapster/contributors","subscribers_url":"https://api.github.com/repos/mwielgus/heapster/subscribers","subscription_url":"https://api.github.com/repos/mwielgus/heapster/subscription","commits_url":"https://api.github.com/repos/mwielgus/heapster/commits{/sha}","git_commits_url":"https://api.github.com/repos/mwielgus/heapster/git/commits{/sha}","comments_url":"https://api.github.com/repos/mwielgus/heapster/comments{/number}","issue_comment_url":"https://api.github.com/repos/mwielgus/heapster/issues/comments{/number}","contents_url":"https://api.github.com/repos/mwielgus/heapster/contents/{+path}","compare_url":"https://api.github.com/repos/mwielgus/heapster/compare/{base}...{head}","merges_url":"https://api.github.com/repos/mwielgus/heapster/merges","archive_url":"https://api.github.com/repos/mwielgus/heapster/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/mwielgus/heapster/downloads","issues_url":"https://api.github.com/repos/mwielgus/heapster/issues{/number}","pulls_url":"https://api.github.com/repos/mwielgus/heapster/pulls{/number}","milestones_url":"https://api.github.com/repos/mwielgus/heapster/milestones{/number}","notifications_url":"https://api.github.com/repos/mwielgus/heapster/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/mwielgus/heapster/labels{/name}","releases_url":"https://api.github.com/repos/mwielgus/heapster/releases{/id}","created_at":"2015-08-07T10:18:26Z","updated_at":"2015-08-07T10:18:27Z","pushed_at":"2015-12-08T13:06:51Z","git_url":"git://github.com/mwielgus/heapster.git","ssh_url":"git@github.com:mwielgus/heapster.git","clone_url":"https://github.com/mwielgus/heapster.git","svn_url":"https://github.com/mwielgus/heapster","homepage":"","size":21707,"stargazers_count":0,"watchers_count":0,"language":"Go","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"kubernetes:master","ref":"master","sha":"f525f2a249940ed4c342ff73e7c9d7b95e920cf9","user":{"login":"kubernetes","id":13629408,"avatar_url":"https://avatars.githubusercontent.com/u/13629408?v=3","gravatar_id":"","url":"https://api.github.com/users/kubernetes","html_url":"https://github.com/kubernetes","followers_url":"https://api.github.com/users/kubernetes/followers","following_url":"https://api.github.com/users/kubernetes/following{/other_user}","gists_url":"https://api.github.com/users/kubernetes/gists{/gist_id}","starred_url":"https://api.github.com/users/kubernetes/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kubernetes/subscriptions","organizations_url":"https://api.github.com/users/kubernetes/orgs","repos_url":"https://api.github.com/users/kubernetes/repos","events_url":"https://api.github.com/users/kubernetes/events{/privacy}","received_events_url":"https://api.github.com/users/kubernetes/received_events","type":"Organization","site_admin":false},"repo":{"id":23272510,"name":"heapster","full_name":"kubernetes/heapster","owner":{"login":"kubernetes","id":13629408,"avatar_url":"https://avatars.githubusercontent.com/u/13629408?v=3","gravatar_id":"","url":"https://api.github.com/users/kubernetes","html_url":"https://github.com/kubernetes","followers_url":"https://api.github.com/users/kubernetes/followers","following_url":"https://api.github.com/users/kubernetes/following{/other_user}","gists_url":"https://api.github.com/users/kubernetes/gists{/gist_id}","starred_url":"https://api.github.com/users/kubernetes/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kubernetes/subscriptions","organizations_url":"https://api.github.com/users/kubernetes/orgs","repos_url":"https://api.github.com/users/kubernetes/repos","events_url":"https://api.github.com/users/kubernetes/events{/privacy}","received_events_url":"https://api.github.com/users/kubernetes/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/kubernetes/heapster","description":"Compute Resource Usage Analysis and Monitoring of Container Clusters","fork":false,"url":"https://api.github.com/repos/kubernetes/heapster","forks_url":"https://api.github.com/repos/kubernetes/heapster/forks","keys_url":"https://api.github.com/repos/kubernetes/heapster/keys{/key_id}","collaborators_url":"https://api.github.com/repos/kubernetes/heapster/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/kubernetes/heapster/teams","hooks_url":"https://api.github.com/repos/kubernetes/heapster/hooks","issue_events_url":"https://api.github.com/repos/kubernetes/heapster/issues/events{/number}","events_url":"https://api.github.com/repos/kubernetes/heapster/events","assignees_url":"https://api.github.com/repos/kubernetes/heapster/assignees{/user}","branches_url":"https://api.github.com/repos/kubernetes/heapster/branches{/branch}","tags_url":"https://api.github.com/repos/kubernetes/heapster/tags","blobs_url":"https://api.github.com/repos/kubernetes/heapster/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/kubernetes/heapster/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/kubernetes/heapster/git/refs{/sha}","trees_url":"https://api.github.com/repos/kubernetes/heapster/git/trees{/sha}","statuses_url":"https://api.github.com/repos/kubernetes/heapster/statuses/{sha}","languages_url":"https://api.github.com/repos/kubernetes/heapster/languages","stargazers_url":"https://api.github.com/repos/kubernetes/heapster/stargazers","contributors_url":"https://api.github.com/repos/kubernetes/heapster/contributors","subscribers_url":"https://api.github.com/repos/kubernetes/heapster/subscribers","subscription_url":"https://api.github.com/repos/kubernetes/heapster/subscription","commits_url":"https://api.github.com/repos/kubernetes/heapster/commits{/sha}","git_commits_url":"https://api.github.com/repos/kubernetes/heapster/git/commits{/sha}","comments_url":"https://api.github.com/repos/kubernetes/heapster/comments{/number}","issue_comment_url":"https://api.github.com/repos/kubernetes/heapster/issues/comments{/number}","contents_url":"https://api.github.com/repos/kubernetes/heapster/contents/{+path}","compare_url":"https://api.github.com/repos/kubernetes/heapster/compare/{base}...{head}","merges_url":"https://api.github.com/repos/kubernetes/heapster/merges","archive_url":"https://api.github.com/repos/kubernetes/heapster/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/kubernetes/heapster/downloads","issues_url":"https://api.github.com/repos/kubernetes/heapster/issues{/number}","pulls_url":"https://api.github.com/repos/kubernetes/heapster/pulls{/number}","milestones_url":"https://api.github.com/repos/kubernetes/heapster/milestones{/number}","notifications_url":"https://api.github.com/repos/kubernetes/heapster/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/kubernetes/heapster/labels{/name}","releases_url":"https://api.github.com/repos/kubernetes/heapster/releases{/id}","created_at":"2014-08-24T03:50:18Z","updated_at":"2015-12-07T16:59:55Z","pushed_at":"2015-12-08T18:21:20Z","git_url":"git://github.com/kubernetes/heapster.git","ssh_url":"git@github.com:kubernetes/heapster.git","clone_url":"https://github.com/kubernetes/heapster.git","svn_url":"https://github.com/kubernetes/heapster","homepage":"","size":23242,"stargazers_count":500,"watchers_count":500,"language":"Go","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":144,"mirror_url":null,"open_issues_count":141,"forks":144,"open_issues":141,"watchers":500,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/kubernetes/heapster/pulls/769"},"html":{"href":"https://github.com/kubernetes/heapster/pull/769"},"issue":{"href":"https://api.github.com/repos/kubernetes/heapster/issues/769"},"comments":{"href":"https://api.github.com/repos/kubernetes/heapster/issues/769/comments"},"review_comments":{"href":"https://api.github.com/repos/kubernetes/heapster/pulls/769/comments"},"review_comment":{"href":"https://api.github.com/repos/kubernetes/heapster/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/kubernetes/heapster/pulls/769/commits"},"statuses":{"href":"https://api.github.com/repos/kubernetes/heapster/statuses/5e14d43c173c1135745248ba60fe8eb0b72003b8"}}}}