{"action":"opened","issue":{"url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/issues/7453","labels_url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/issues/7453/labels{/name}","comments_url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/issues/7453/comments","events_url":"https://api.github.com/repos/GoogleCloudPlatform/kubernetes/issues/7453/events","html_url":"https://github.com/GoogleCloudPlatform/kubernetes/issues/7453","id":71678615,"number":7453,"title":"Flaky e2e test: Services should provide DNS for the cluster - Expected <int>: 3 to equal <int>: 0","user":{"login":"quinton-hoole","id":10390785,"avatar_url":"https://avatars.githubusercontent.com/u/10390785?v=3","gravatar_id":"","url":"https://api.github.com/users/quinton-hoole","html_url":"https://github.com/quinton-hoole","followers_url":"https://api.github.com/users/quinton-hoole/followers","following_url":"https://api.github.com/users/quinton-hoole/following{/other_user}","gists_url":"https://api.github.com/users/quinton-hoole/gists{/gist_id}","starred_url":"https://api.github.com/users/quinton-hoole/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/quinton-hoole/subscriptions","organizations_url":"https://api.github.com/users/quinton-hoole/orgs","repos_url":"https://api.github.com/users/quinton-hoole/repos","events_url":"https://api.github.com/users/quinton-hoole/events{/privacy}","received_events_url":"https://api.github.com/users/quinton-hoole/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2015-04-28T19:24:04Z","updated_at":"2015-04-28T19:24:04Z","closed_at":null,"body":"This test fails intermittently on our CI system, and it's fairly easy to reproduce the error in a local test cluster, which I've done.\r\n\r\nThe error message output by the e2e test is consistently:\r\n\r\n/go/src/github.com/GoogleCloudPlatform/kubernetes/_output/dockerized/go/src/github.com/GoogleCloudPlatform/kubernetes/test/e2e/service.go:165\r\nExpected\r\n    <int>: 3\r\nto equal\r\n    <int>: 0\r\n\r\nDigging a bit deeper, it looks like our kube2sky bridge is failing to GET the pods for which name resolution is being requested:\r\n\r\nLookup using dns-test-3c04e84a-eda2-11e4-ab38-42010af01555 for kubernetes-ro failed: the server could not find the requested resource (get pods dns-test-3c04e84a-eda2-11e4-ab38-42010af01555)\r\nLookup using dns-test-3c04e84a-eda2-11e4-ab38-42010af01555 for kubernetes-ro.default failed: the server could not find the requested resource (get pods dns-test-3c04e84a-eda2-11e4-ab38-42010af01555)\r\nLookup using dns-test-3c04e84a-eda2-11e4-ab38-42010af01555 for kubernetes-ro.default.kubernetes.local failed: the server could not find the requested resource (get pods dns-test-3c04e84a-eda2-11e4-ab38-42010af01555)\r\nlookups using dns-test-3c04e84a-eda2-11e4-ab38-42010af01555 failed for: [kubernetes-ro kubernetes-ro.default kubernetes-ro.default.kubernetes.local]\r\n\r\nWhile kube2sky was busy returning these errors, I confirmed that I was able to GET the above PODs using kubectl, so the API is doing the right thing. So this looks like a bug in kube2sky.\r\n\r\nI'll keep digging. This is seemingly breaking quite a few other e2e tests which rely on cluster DNS (understandably).  For example, when this test fails, the following other tests also seem to fail in concert:\r\n\r\nkubectl guestbook should create and stop a working application\r\n/go/src/github.com/GoogleCloudPlatform/kubernetes/_output/dockerized/go/src/github.com/GoogleCloudPlatform/kubernetes/test/e2e/kubectl.go:113\r\nFrontend service did not start serving content in 600 seconds.\r\n\r\nEvents should be sent by kubelets and the scheduler about pods scheduling and running\r\n/go/src/github.com/GoogleCloudPlatform/kubernetes/_output/dockerized/go/src/github.com/GoogleCloudPlatform/kubernetes/test/e2e/events.go:124\r\nkubelet events from running pod\r\nExpected\r\n    <int>: 0\r\nnot to be zero-valued\r\n\r\nCluster level logging using Elasticsearch should check that logs from pods on all nodes are ingested into Elasticsearch\r\n/go/src/github.com/GoogleCloudPlatform/kubernetes/_output/dockerized/go/src/github.com/GoogleCloudPlatform/kubernetes/test/e2e/es_cluster_logging.go:46\r\nFailed to find all 200 log lines\r\n\r\n\r\nAside: #4852 (health checks for DNS) would probably also help to prevent this failure mode."}}