{"action":"created","issue":{"url":"https://api.github.com/repos/kubernetes/kubernetes/issues/14695","labels_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/14695/labels{/name}","comments_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/14695/comments","events_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/14695/events","html_url":"https://github.com/kubernetes/kubernetes/issues/14695","id":108770562,"number":14695,"title":"flaky e2e: Network when a minion node becomes unreachable [replication controller] recreates pods scheduled on the unreachable minion node AND allows scheduling of pods on a minion after it rejoins the cluster","user":{"login":"goltermann","id":9358478,"avatar_url":"https://avatars.githubusercontent.com/u/9358478?v=3","gravatar_id":"","url":"https://api.github.com/users/goltermann","html_url":"https://github.com/goltermann","followers_url":"https://api.github.com/users/goltermann/followers","following_url":"https://api.github.com/users/goltermann/following{/other_user}","gists_url":"https://api.github.com/users/goltermann/gists{/gist_id}","starred_url":"https://api.github.com/users/goltermann/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/goltermann/subscriptions","organizations_url":"https://api.github.com/users/goltermann/orgs","repos_url":"https://api.github.com/users/goltermann/repos","events_url":"https://api.github.com/users/goltermann/events{/privacy}","received_events_url":"https://api.github.com/users/goltermann/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/kubernetes/kubernetes/labels/area/cluster-lifecycle","name":"area/cluster-lifecycle","color":"d4c5f9"},{"url":"https://api.github.com/repos/kubernetes/kubernetes/labels/kind/flake","name":"kind/flake","color":"f7c6c7"},{"url":"https://api.github.com/repos/kubernetes/kubernetes/labels/priority/P1","name":"priority/P1","color":"eb6420"}],"state":"open","locked":false,"assignee":{"login":"fgrzadkowski","id":10820726,"avatar_url":"https://avatars.githubusercontent.com/u/10820726?v=3","gravatar_id":"","url":"https://api.github.com/users/fgrzadkowski","html_url":"https://github.com/fgrzadkowski","followers_url":"https://api.github.com/users/fgrzadkowski/followers","following_url":"https://api.github.com/users/fgrzadkowski/following{/other_user}","gists_url":"https://api.github.com/users/fgrzadkowski/gists{/gist_id}","starred_url":"https://api.github.com/users/fgrzadkowski/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fgrzadkowski/subscriptions","organizations_url":"https://api.github.com/users/fgrzadkowski/orgs","repos_url":"https://api.github.com/users/fgrzadkowski/repos","events_url":"https://api.github.com/users/fgrzadkowski/events{/privacy}","received_events_url":"https://api.github.com/users/fgrzadkowski/received_events","type":"User","site_admin":false},"milestone":null,"comments":17,"created_at":"2015-09-29T00:07:26Z","updated_at":"2015-10-07T13:28:26Z","closed_at":null,"body":"Nodes Network when a minion node becomes unreachable [replication controller] recreates pods scheduled on the unreachable minion node AND allows scheduling of pods on a minion after it rejoins the cluster(Failed 17 times in the last 30 runs. Stability: 43 %)Failed with 0 retries"},"comment":{"url":"https://api.github.com/repos/kubernetes/kubernetes/issues/comments/146196322","html_url":"https://github.com/kubernetes/kubernetes/issues/14695#issuecomment-146196322","issue_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/14695","id":146196322,"user":{"login":"fgrzadkowski","id":10820726,"avatar_url":"https://avatars.githubusercontent.com/u/10820726?v=3","gravatar_id":"","url":"https://api.github.com/users/fgrzadkowski","html_url":"https://github.com/fgrzadkowski","followers_url":"https://api.github.com/users/fgrzadkowski/followers","following_url":"https://api.github.com/users/fgrzadkowski/following{/other_user}","gists_url":"https://api.github.com/users/fgrzadkowski/gists{/gist_id}","starred_url":"https://api.github.com/users/fgrzadkowski/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fgrzadkowski/subscriptions","organizations_url":"https://api.github.com/users/fgrzadkowski/orgs","repos_url":"https://api.github.com/users/fgrzadkowski/repos","events_url":"https://api.github.com/users/fgrzadkowski/events{/privacy}","received_events_url":"https://api.github.com/users/fgrzadkowski/received_events","type":"User","site_admin":false},"created_at":"2015-10-07T13:28:26Z","updated_at":"2015-10-07T13:28:26Z","body":"It seems we have a new type of failure:\r\n```\r\n17:46:05 • Failure [650.683 seconds]\r\n17:46:05 Nodes\r\n17:46:05 /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/resize_nodes.go:539\r\n17:46:05   Network\r\n17:46:05   /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/resize_nodes.go:538\r\n17:46:05     when a minion node becomes unreachable\r\n17:46:05     /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/resize_nodes.go:537\r\n17:46:05       [replication controller] recreates pods scheduled on the unreachable minion node AND allows scheduling of pods on a minion after it rejoins the cluster [It]\r\n17:46:05       /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/resize_nodes.go:536\r\n17:46:05 \r\n17:46:05       Expected error:\r\n17:46:05           <*errors.errorString | 0xc2084fc520>: {\r\n17:46:05               s: \"failed to wait for pods running: [gave up waiting for pod 'additionalpod' to be 'running' after 5m0s]\",\r\n17:46:05           }\r\n17:46:05           failed to wait for pods running: [gave up waiting for pod 'additionalpod' to be 'running' after 5m0s]\r\n17:46:05       not to have occurred\r\n17:46:05 \r\n17:46:05       /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/resize_nodes.go:526\r\n```\r\nhttp://kubekins.dls.corp.google.com/job/kubernetes-e2e-gce-flaky/5644/consoleFull\r\n\r\nIt seems that kubelet on a machine where network is blocked doesn't notice new pod. After reading logs I assume that kubelet doesn't notice that the watch is broken and doesn't try to relist everything. It notices new pod only after watch timeout."}}