{"action":"created","issue":{"url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/issues/7451","labels_url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/issues/7451/labels{/name}","comments_url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/issues/7451/comments","events_url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/issues/7451/events","html_url":"https://github.com/GoogleCloudPlatform/kubernetes/issues/7451","id":71671546,"number":7451,"title":"Hundreds of open connections to etcd.","user":{"login":"timothysc","id":169553,"avatar_url":"https://avatars.githubusercontent.com/u/169553?v=3","gravatar_id":"","url":"https://api.github.com/users/timothysc","html_url":"https://github.com/timothysc","followers_url":"https://api.github.com/users/timothysc/followers","following_url":"https://api.github.com/users/timothysc/following{/other_user}","gists_url":"https://api.github.com/users/timothysc/gists{/gist_id}","starred_url":"https://api.github.com/users/timothysc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/timothysc/subscriptions","organizations_url":"https://api.github.com/users/timothysc/orgs","repos_url":"https://api.github.com/users/timothysc/repos","events_url":"https://api.github.com/users/timothysc/events{/privacy}","received_events_url":"https://api.github.com/users/timothysc/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/labels/priority/P2","name":"priority/P2","color":"fbca04"},{"url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/labels/team/master","name":"team/master","color":"d2b48c"}],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":12,"created_at":"2015-04-28T18:50:37Z","updated_at":"2015-05-31T19:22:30Z","closed_at":"2015-04-28T21:07:59Z","body":"Observations: \r\n\r\nWhen doing nothing on an unloaded system with 23 nodes checking in, we have \r\n~126 open connections to etcd.   \r\n\r\nWhen we start to load the system easily jumps to 400+.  \r\n\r\n`lsof -p $(pgrep kube-apiserver)`\r\n\r\nOther Notes: \r\n* http://stackoverflow.com/questions/17948827/reusing-http-connections-in-golang \r\n* I suspect issues in how we are using the etcd client library, which is currently not thread safe but we are using it in a multiplexed fashion.  \r\n\r\n@wojtek-t @fgrzadkowski @smarterclayton "},"comment":{"url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/issues/comments/107238270","html_url":"https://github.com/GoogleCloudPlatform/kubernetes/issues/7451#issuecomment-107238270","issue_url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/issues/7451","id":107238270,"user":{"login":"davidopp","id":8264113,"avatar_url":"https://avatars.githubusercontent.com/u/8264113?v=3","gravatar_id":"","url":"https://api.github.com/users/davidopp","html_url":"https://github.com/davidopp","followers_url":"https://api.github.com/users/davidopp/followers","following_url":"https://api.github.com/users/davidopp/following{/other_user}","gists_url":"https://api.github.com/users/davidopp/gists{/gist_id}","starred_url":"https://api.github.com/users/davidopp/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/davidopp/subscriptions","organizations_url":"https://api.github.com/users/davidopp/orgs","repos_url":"https://api.github.com/users/davidopp/repos","events_url":"https://api.github.com/users/davidopp/events{/privacy}","received_events_url":"https://api.github.com/users/davidopp/received_events","type":"User","site_admin":false},"created_at":"2015-05-31T19:22:30Z","updated_at":"2015-05-31T19:22:30Z","body":"Should we file a post-1.0 issue for apiserver to merge watches, and perhaps do other kinds of batching?"}}