{"action":"created","issue":{"url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/issues/3939","labels_url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/issues/3939/labels{/name}","comments_url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/issues/3939/comments","events_url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/issues/3939/events","html_url":"https://github.com/GoogleCloudPlatform/kubernetes/issues/3939","id":55938374,"number":3939,"title":"apiserver should clarify which peers it's unable to contact","user":{"login":"alex-mohr","id":8729097,"avatar_url":"https://avatars.githubusercontent.com/u/8729097?v=3","gravatar_id":"","url":"https://api.github.com/users/alex-mohr","html_url":"https://github.com/alex-mohr","followers_url":"https://api.github.com/users/alex-mohr/followers","following_url":"https://api.github.com/users/alex-mohr/following{/other_user}","gists_url":"https://api.github.com/users/alex-mohr/gists{/gist_id}","starred_url":"https://api.github.com/users/alex-mohr/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/alex-mohr/subscriptions","organizations_url":"https://api.github.com/users/alex-mohr/orgs","repos_url":"https://api.github.com/users/alex-mohr/repos","events_url":"https://api.github.com/users/alex-mohr/events{/privacy}","received_events_url":"https://api.github.com/users/alex-mohr/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/labels/area/hosting","name":"area/hosting","color":"0052cc"},{"url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/labels/area/usability","name":"area/usability","color":"00cc00"},{"url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/labels/component/apiserver","name":"component/apiserver","color":"0052cc"},{"url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/labels/priority/P1","name":"priority/P1","color":"eb6420"}],"state":"closed","locked":false,"assignee":{"login":"davidopp","id":8264113,"avatar_url":"https://avatars.githubusercontent.com/u/8264113?v=3","gravatar_id":"","url":"https://api.github.com/users/davidopp","html_url":"https://github.com/davidopp","followers_url":"https://api.github.com/users/davidopp/followers","following_url":"https://api.github.com/users/davidopp/following{/other_user}","gists_url":"https://api.github.com/users/davidopp/gists{/gist_id}","starred_url":"https://api.github.com/users/davidopp/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/davidopp/subscriptions","organizations_url":"https://api.github.com/users/davidopp/orgs","repos_url":"https://api.github.com/users/davidopp/repos","events_url":"https://api.github.com/users/davidopp/events{/privacy}","received_events_url":"https://api.github.com/users/davidopp/received_events","type":"User","site_admin":false},"milestone":null,"comments":2,"created_at":"2015-01-29T18:36:31Z","updated_at":"2015-02-02T23:31:43Z","closed_at":"2015-02-02T23:31:43Z","body":"@davidopp for apiserver.\r\nI had a cluster where etcd became nonfunctional.  The apiserver's error messages weren't amazingly helpful in figuring out which component was having issues -- it should report which peer isn't contactable.\r\n\r\n% gcloud preview container pods --cluster=${PROJECT} list\r\nERROR: F0129 10:20:43.796366   29535 get.go:158] 501: All the given peers are not reachable (Tried to connect to each peer twice and failed) [0]\r\n\r\nSeparate issue tracking kubectl reporting #3937.  Part of long-running clusters #3936.\r\n\r\nExtended log from /var/log/kube-apiserver.log\r\nI0129 18:33:10.977679    5006 handlers.go:104] GET /api/v1beta1/pods?fields=DesiredState.Host%21%3D: (634.881us) 500\r\ngoroutine 2136125 [running]:\r\ngithub.com/GoogleCloudPlatform/kubernetes/pkg/httplog.(*respLogger).WriteHeader(0xc20b3783c0, 0x1f4)\r\n        /go/src/github.com/GoogleCloudPlatform/kubernetes/_output/dockerized/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/httplog/log.go:182 +0x9c\r\ngithub.com/GoogleCloudPlatform/kubernetes/pkg/apiserver.writeJSON(0x1f4, 0x7f7ce12efa78, 0xc208044b20, 0x7f7ce12efc18, 0xc20c593500, 0x7f7ce13049a8, 0xc20b3783c0)\r\n        /go/src/github.com/GoogleCloudPlatform/kubernetes/_output/dockerized/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/apiserver/apiserver.go:307 +0x239\r\ngithub.com/GoogleCloudPlatform/kubernetes/pkg/apiserver.errorJSON(0x7f7ce12f2a50, 0xc20d097aa0, 0x7f7ce12efa78, 0xc208044b20, 0x7f7ce13049a8, 0xc20b3783c0)\r\n        /go/src/github.com/GoogleCloudPlatform/kubernetes/_output/dockerized/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/apiserver/apiserver.go:314 +0xa4\r\ngithub.com/GoogleCloudPlatform/kubernetes/pkg/apiserver.(*RESTHandler).handleRESTStorage(0xc20801b7c0, 0xc20df06130, 0x1, 0x1, 0xc20a5544e0, 0x7f7ce13049a8, 0xc20b3783c0, 0x7f7ce12f2b28, 0xc208045ea0, 0xb0fef0, ...)\r\n        /go/src/github.com/GoogleCloudPlatform/kubernetes/_output/dockerized/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/apiserver/resthandler.go:169 +0x797\r\ngithub.com/GoogleCloudPlatform/kubernetes/pkg/apiserver.(*RESTHandler).ServeHTTP(0xc20801b7c0, 0x7f7ce13049a8, 0xc20b3783c0, 0xc20a5544e0)\r\n        /go/src/github.com/GoogleCloudPlatform/kubernetes/_output/dockerized/go/src/github.com/GoogleCloudPlatform/kubernetes/pkg/apiserver/resthandler.go:56 +0x2fc\r\nnet/http.func·012(0x7f7ce13049a8, 0xc20b3783c0, 0xc20a5544e0)\r\n        /usr/src/go/src/pkg/net/http/server.go:1267 +0xc4\r\nnet/http.HandlerFunc.ServeHTTP(0xc208118440, 0x7f7ce13049a8, 0xc20b3783c0, 0xc20a5544e0)\r\n        /usr/src/go/src/pkg/net/http/server.go:1235 +0x40\r\ngithub.com/GoogleCloudPlatform/kubernetes/pkg/apiserver.func·001(0xc20d097680, 0xc20a7814f0)\r\n        /go/src/github.com/GoogleCloudPlatform/kubernetes/_output/dockerized/go/src/github.com/GoogleCloudPlatform/kubernetes\r\nI0129 18:33:10.978652    5006 errors.go:52] An unchecked error was received: 501: All the given peers are not reachable (Tried to connect to each peer twice and failed) [0]"},"comment":{"url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/issues/comments/72561759","html_url":"https://github.com/GoogleCloudPlatform/kubernetes/issues/3939#issuecomment-72561759","issue_url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/issues/3939","id":72561759,"user":{"login":"brendandburns","id":5751682,"avatar_url":"https://avatars.githubusercontent.com/u/5751682?v=3","gravatar_id":"","url":"https://api.github.com/users/brendandburns","html_url":"https://github.com/brendandburns","followers_url":"https://api.github.com/users/brendandburns/followers","following_url":"https://api.github.com/users/brendandburns/following{/other_user}","gists_url":"https://api.github.com/users/brendandburns/gists{/gist_id}","starred_url":"https://api.github.com/users/brendandburns/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/brendandburns/subscriptions","organizations_url":"https://api.github.com/users/brendandburns/orgs","repos_url":"https://api.github.com/users/brendandburns/repos","events_url":"https://api.github.com/users/brendandburns/events{/privacy}","received_events_url":"https://api.github.com/users/brendandburns/received_events","type":"User","site_admin":false},"created_at":"2015-02-02T23:31:42Z","updated_at":"2015-02-02T23:31:42Z","body":"This is straight from etcd.  Created https://github.com/GoogleCloudPlatform/kubernetes/issues/4036 and closing this."}}