{"action":"created","issue":{"url":"https://api.github.com/repos/kubernetes/kubernetes/issues/12379","labels_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/12379/labels{/name}","comments_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/12379/comments","events_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/12379/events","html_url":"https://github.com/kubernetes/kubernetes/issues/12379","id":99578838,"number":12379,"title":"kube-apiserver 1.0.2 - \"Unable to perform initial IP allocation check\"","user":{"login":"MrFishFinger","id":4768596,"avatar_url":"https://avatars.githubusercontent.com/u/4768596?v=3","gravatar_id":"","url":"https://api.github.com/users/MrFishFinger","html_url":"https://github.com/MrFishFinger","followers_url":"https://api.github.com/users/MrFishFinger/followers","following_url":"https://api.github.com/users/MrFishFinger/following{/other_user}","gists_url":"https://api.github.com/users/MrFishFinger/gists{/gist_id}","starred_url":"https://api.github.com/users/MrFishFinger/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/MrFishFinger/subscriptions","organizations_url":"https://api.github.com/users/MrFishFinger/orgs","repos_url":"https://api.github.com/users/MrFishFinger/repos","events_url":"https://api.github.com/users/MrFishFinger/events{/privacy}","received_events_url":"https://api.github.com/users/MrFishFinger/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/kubernetes/kubernetes/labels/kind/bug","name":"kind/bug","color":"e11d21"},{"url":"https://api.github.com/repos/kubernetes/kubernetes/labels/priority/P2","name":"priority/P2","color":"fbca04"},{"url":"https://api.github.com/repos/kubernetes/kubernetes/labels/team/control-plane","name":"team/control-plane","color":"d2b48c"},{"url":"https://api.github.com/repos/kubernetes/kubernetes/labels/team/node","name":"team/node","color":"d2b48c"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2015-08-07T04:54:05Z","updated_at":"2015-08-17T10:30:37Z","closed_at":null,"body":"Kubernetes release 1.0.2 on a couple of DigitialOcean CoreOS stable 723.3.0 machines with a cloud-config.yaml.\r\n\r\nI am having lots of issues with kube-apiserver crashing on the master machine:\r\n\r\n\r\n**kube-apiserver errors:**\r\n```\r\nAug 07 04:52:03 master kube-apiserver[5792]: I0807 04:52:03.860951    5792 plugins.go:69] No cloud provider specified.\r\nAug 07 04:52:03 master kube-apiserver[5792]: I0807 04:52:03.862682    5792 master.go:273] Node port range unspecified. Defaulting to 30000-32767.\r\nAug 07 04:52:03 master kube-apiserver[5792]: [restful] 2015/08/07 04:52:03 log.go:30: [restful/swagger] listing is available at https://46.101.171.12:6443/swaggerapi/\r\nAug 07 04:52:03 master kube-apiserver[5792]: [restful] 2015/08/07 04:52:03 log.go:30: [restful/swagger] https://46.101.171.12:6443/swaggerui/ is mapped to folder /swagger-ui/\r\nAug 07 04:52:14 master kube-apiserver[5792]: F0807 04:52:14.220342    5792 controller.go:80] Unable to perform initial IP allocation check: unable to refresh the service IP block: 501: All the given peers are not reachable (failed to propose on members [http://localhost:2379] twice [last error: Get http://localhost:2379/v2/keys/registry/ranges/serviceips?quorum=false&recursive=false&sorted=false: dial tcp: lookup localhost: no such host]) [0]\r\n```\r\n\r\n**kube-controller-manager and kube-scheduler:**\r\n*(which are likely caused by kube-apiserver being unavailable on port 8080)*\r\n```\r\nAug 07 05:05:23 master2 kube-controller-manager[918]: E0807 05:05:23.124385     918 reflector.go:136] Failed to list *api.PersistentVolume: Get http://127.0.0.1:8080/api/v1/persistentvolumes: dial tcp 127.0.0.1:8080: connection refused\r\nAug 07 05:05:23 master2 kube-controller-manager[918]: E0807 05:05:23.125390     918 reflector.go:136] Failed to list *api.PersistentVolumeClaim: Get http://127.0.0.1:8080/api/v1/persistentvolumeclaims: dial tcp 127.0.0.1:8080: connection refused\r\nAug 07 05:05:23 master2 kube-controller-manager[918]: E0807 05:05:23.125422     918 reflector.go:136] Failed to list *api.PersistentVolume: Get http://127.0.0.1:8080/api/v1/persistentvolumes: dial tcp 127.0.0.1:8080: connection refused\r\n```\r\n```\r\nAug 07 05:06:51 master2 kube-scheduler[1181]: E0807 05:06:51.812923    1181 reflector.go:136] Failed to list *api.Node: Get http://127.0.0.1:8080/api/v1/nodes?fieldSelector=spec.unschedulable%3Dfalse: dial tcp 127.0.0.1:8080: connection refused\r\nAug 07 05:06:51 master2 kube-scheduler[1181]: E0807 05:06:51.813797    1181 reflector.go:136] Failed to list *api.Pod: Get http://127.0.0.1:8080/api/v1/pods?fieldSelector=spec.nodeName%21%3D: dial tcp 127.0.0.1:8080: connection refused\r\nAug 07 05:06:51 master2 kube-scheduler[1181]: E0807 05:06:51.813863    1181 reflector.go:136] Failed to list *api.Pod: Get http://127.0.0.1:8080/api/v1/pods?fieldSelector=spec.nodeName%3D: dial tcp 127.0.0.1:8080: connection refused\r\nAug 07 05:06:51 master2 kube-scheduler[1181]: E0807 05:06:51.814495    1181 reflector.go:136] Failed to list *api.Service: Get http://127.0.0.1:8080/api/v1/services: dial tcp 127.0.0.1:8080: connection refused\r\n```\r\n\r\n\r\n**Cloud-config file for master**\r\n```yaml\r\n#cloud-config\r\n\r\ncoreos:\r\n  etcd2:\r\n    # https://discovery.etcd.io/new?size=2\r\n    #discovery: https://discovery.etcd.io/6b54b1f7e80e49de49e6440256cbb937\r\n    name: master\r\n    initial-cluster-token: k8s_etcd\r\n    initial-cluster: master=http://$public_ipv4:2380\r\n    advertise-client-urls: http://$public_ipv4:2379\r\n    initial-advertise-peer-urls: http://$public_ipv4:2380\r\n    listen-client-urls: http://0.0.0.0:2379\r\n    listen-peer-urls: http://$public_ipv4:2380\r\n  fleet:\r\n    public-ip: $public_ipv4   # used for fleetctl ssh command\r\n    etcd_servers: http://127.0.0.1:2379\r\n    metadata: k8srole=master\r\n  flannel:\r\n    etcd_endpoints: http://localhost:2379\r\n  locksmithd:\r\n    endpoint: http://localhost:2379\r\n  units:\r\n    - name: etcd2.service\r\n      command: start\r\n    - name: fleet.service\r\n      command: start\r\n    - name: flanneld.service\r\n      drop-ins:\r\n        - name: 50-network-config.conf\r\n          content: |\r\n            [Service]\r\n            ExecStartPre=/usr/bin/etcdctl set /coreos.com/network/config '{ \"Network\": \"10.100.0.0/16\", \"Backend\": {\"Type\": \"vxlan\"} }'\r\n      command: start\r\n    - name: get-kubectl.service\r\n      command: start\r\n      content: |\r\n        [Unit]\r\n        Description=Get kubectl client tool\r\n        Documentation=https://github.com/GoogleCloudPlatform/kubernetes\r\n        Requires=network-online.target\r\n        After=network-online.target\r\n        [Service]\r\n        ExecStart=/usr/bin/wget -N -P /opt/bin https://storage.googleapis.com/kubernetes-release/release/v1.0.2/bin/linux/amd64/kubectl\r\n        ExecStart=/usr/bin/chmod +x /opt/bin/kubectl\r\n        Type=oneshot\r\n        RemainAfterExit=true\r\n    - name: kube-apiserver.service\r\n      command: start\r\n      content: |\r\n        [Unit]\r\n        Description=Kubernetes API Server\r\n        Documentation=https://github.com/GoogleCloudPlatform/kubernetes\r\n        ######### Requires=etcd2-waiter.service\r\n        ######### After=etcd2-waiter.service\r\n        [Service]\r\n        ExecStartPre=/usr/bin/wget -N -P /opt/bin https://storage.googleapis.com/kubernetes-release/release/v1.0.2/bin/linux/amd64/kube-apiserver\r\n        ExecStartPre=/usr/bin/chmod +x /opt/bin/kube-apiserver\r\n        ExecStart=/opt/bin/kube-apiserver \\\r\n        --insecure-bind-address=0.0.0.0 \\\r\n        --service-cluster-ip-range=10.100.0.0/16 \\\r\n        --etcd-servers=http://localhost:2379 \\\r\n        --bind-address=$public_ipv4\r\n        Restart=always\r\n        RestartSec=10\r\n    - name: kube-controller-manager.service\r\n      command: start\r\n      content: |\r\n        [Unit]\r\n        Description=Kubernetes Controller Manager\r\n        Documentation=https://github.com/GoogleCloudPlatform/kubernetes\r\n        Requires=kube-apiserver.service\r\n        After=kube-apiserver.service\r\n        [Service]\r\n        ExecStartPre=/usr/bin/wget -N -P /opt/bin https://storage.googleapis.com/kubernetes-release/release/v1.0.2/bin/linux/amd64/kube-controller-manager\r\n        ExecStartPre=/usr/bin/chmod +x /opt/bin/kube-controller-manager\r\n        ExecStart=/opt/bin/kube-controller-manager \\\r\n        --master=127.0.0.1:8080\r\n        Restart=always\r\n        RestartSec=10\r\n    - name: kube-scheduler.service\r\n      command: start\r\n      content: |\r\n        [Unit]\r\n        Description=Kubernetes Scheduler\r\n        Documentation=https://github.com/GoogleCloudPlatform/kubernetes\r\n        Requires=kube-apiserver.service\r\n        After=kube-apiserver.service\r\n        [Service]\r\n        ExecStartPre=/usr/bin/wget -N -P /opt/bin https://storage.googleapis.com/kubernetes-release/release/v1.0.2/bin/linux/amd64/kube-scheduler\r\n        ExecStartPre=/usr/bin/chmod +x /opt/bin/kube-scheduler\r\n        ExecStart=/opt/bin/kube-scheduler \\\r\n        --master=127.0.0.1:8080\r\n        Restart=always\r\n        RestartSec=10\r\n    - name: kube-register.service\r\n      command: start\r\n      content: |\r\n        [Unit]\r\n        Description=Kubernetes Registration Service\r\n        Documentation=https://github.com/kelseyhightower/kube-register\r\n        Requires=kube-apiserver.service fleet.service\r\n        After=kube-apiserver.service fleet.service\r\n        [Service]\r\n        ExecStartPre=-/usr/bin/wget -nc -O /opt/bin/kube-register https://github.com/kelseyhightower/kube-register/releases/download/v0.0.4/kube-register-0.0.4-linux-amd64\r\n        ExecStartPre=/usr/bin/chmod +x /opt/bin/kube-register\r\n        ExecStart=/opt/bin/kube-register \\\r\n        --metadata=k8srole=node \\\r\n        --fleet-endpoint=unix:///var/run/fleet.sock \\\r\n        --api-endpoint=http://127.0.0.1:8080\r\n        Restart=always\r\n        RestartSec=10\r\n```"},"comment":{"url":"https://api.github.com/repos/kubernetes/kubernetes/issues/comments/131771155","html_url":"https://github.com/kubernetes/kubernetes/issues/12379#issuecomment-131771155","issue_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/12379","id":131771155,"user":{"login":"kgrvamsi","id":1199031,"avatar_url":"https://avatars.githubusercontent.com/u/1199031?v=3","gravatar_id":"","url":"https://api.github.com/users/kgrvamsi","html_url":"https://github.com/kgrvamsi","followers_url":"https://api.github.com/users/kgrvamsi/followers","following_url":"https://api.github.com/users/kgrvamsi/following{/other_user}","gists_url":"https://api.github.com/users/kgrvamsi/gists{/gist_id}","starred_url":"https://api.github.com/users/kgrvamsi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kgrvamsi/subscriptions","organizations_url":"https://api.github.com/users/kgrvamsi/orgs","repos_url":"https://api.github.com/users/kgrvamsi/repos","events_url":"https://api.github.com/users/kgrvamsi/events{/privacy}","received_events_url":"https://api.github.com/users/kgrvamsi/received_events","type":"User","site_admin":false},"created_at":"2015-08-17T10:30:37Z","updated_at":"2015-08-17T10:30:37Z","body":"I see this behavior when i run the kube-api server inside a docker container\r\n\r\n    F0817 10:28:11.294801      82 controller.go:80] Unable to perform initial IP allocation check: unable to persist the updated service IP allocations: 502:  (unhandled http status [Moved Permanently] with body []) [0]\r\n  "}}