{"action":"created","issue":{"url":"https://api.github.com/repos/coreos/flannel/issues/218","labels_url":"https://api.github.com/repos/coreos/flannel/issues/218/labels{/name}","comments_url":"https://api.github.com/repos/coreos/flannel/issues/218/comments","events_url":"https://api.github.com/repos/coreos/flannel/issues/218/events","html_url":"https://github.com/coreos/flannel/issues/218","id":90126757,"number":218,"title":"Question: How to setup a mini cluster with flannel on ec2","user":{"login":"misakwa","id":1142616,"avatar_url":"https://avatars.githubusercontent.com/u/1142616?v=3","gravatar_id":"","url":"https://api.github.com/users/misakwa","html_url":"https://github.com/misakwa","followers_url":"https://api.github.com/users/misakwa/followers","following_url":"https://api.github.com/users/misakwa/following{/other_user}","gists_url":"https://api.github.com/users/misakwa/gists{/gist_id}","starred_url":"https://api.github.com/users/misakwa/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/misakwa/subscriptions","organizations_url":"https://api.github.com/users/misakwa/orgs","repos_url":"https://api.github.com/users/misakwa/repos","events_url":"https://api.github.com/users/misakwa/events{/privacy}","received_events_url":"https://api.github.com/users/misakwa/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":22,"created_at":"2015-06-22T15:08:08Z","updated_at":"2015-06-23T02:54:54Z","closed_at":null,"body":"I have been experiencing some issues setting up a small cluster on ec2 from following the docs.\r\n\r\nSpecs:\r\nOS: CoreOS stable (681.2.0) - ami-5f2f5528\r\nEtcd: etcd version 2.0.10\r\nFleet: fleetd version 0.10.2\r\nFlannel: 0.4.0\r\nInstance type: t2.micro\r\n\r\nHere is my cloud init config:\r\n\r\n```\r\n#cloud-config\r\n\r\n---\r\ncoreos:\r\n  etcd2:\r\n    advertise-client-urls: http://$private_ipv4:2379\r\n    initial-advertise-peer-urls: http://$private_ipv4:2380\r\n    listen-client-urls: http://0.0.0.0:2379,http://0.0.0.0:4001\r\n    listen-peer-urls: http://$private_ipv4:2380,http://0.0.0.0:7001\r\n    discovery: https://discovery.etcd.io/<my-token>\r\n  fleet:\r\n    public-ip: $private_ipv4\r\n    metadata: role=services,disk=ssd,region=us-east,provider=amazon\r\n  units:\r\n    - name: etcd2.service\r\n      command: start\r\n    - name: fleet.service\r\n      command: start\r\n    - name: flanneld.service\r\n      drop-ins:\r\n      - name: 50-network-config.conf\r\n        content: |\r\n          [Service]\r\n          ExecStartPre=/usr/bin/etcdctl set /coreos.com/network/config '{ \"Network\": \"10.1.0.0/16\" }'\r\n      command: start\r\n    - name: docker-tcp.socket\r\n      command: start\r\n      enable: true\r\n      content: |\r\n        [Unit]\r\n        Description=Docker Socket for the API\r\n\r\n        [Socket]\r\n        ListenStream=2375\r\n        Service=docker.service\r\n        BindIPv6Only=both\r\n\r\n        [Install]\r\n        WantedBy=sockets.target\r\n```\r\n\r\nI spun up 9 instances (using an auto scaling group) all running with the same cloud init config (from a launch config/template).\r\njournalctl shows:\r\n\r\n```\r\nJun 22 13:10:57 ip-172-31-36-121.ec2.internal sdnotify-proxy[4388]: Pulling repository quay.io/coreos/flannel\r\nJun 22 13:10:57 ip-172-31-36-121.ec2.internal sdnotify-proxy[4388]: 73b8daf50cec: Pulling image (0.4.0) from quay.io/coreos/flannel\r\nJun 22 13:10:57 ip-172-31-36-121.ec2.internal sdnotify-proxy[4388]: 73b8daf50cec: Pulling image (0.4.0) from quay.io/coreos/flannel, endpoint: https://quay.io/v1/\r\nJun 22 13:10:57 ip-172-31-36-121.ec2.internal sdnotify-proxy[4388]: 73b8daf50cec: Pulling dependent layers\r\nJun 22 13:10:57 ip-172-31-36-121.ec2.internal sdnotify-proxy[4388]: 91a6195f52a2: Pulling metadata\r\nJun 22 13:10:58 ip-172-31-36-121.ec2.internal sdnotify-proxy[4388]: 91a6195f52a2: Pulling fs layer\r\nJun 22 13:10:58 ip-172-31-36-121.ec2.internal sdnotify-proxy[4388]: 91a6195f52a2: Error pulling dependent layers\r\nJun 22 13:10:58 ip-172-31-36-121.ec2.internal sdnotify-proxy[4388]: 73b8daf50cec: Error pulling image (0.4.0) from quay.io/coreos/flannel, endpoint: https://quay.io/v1/, Server error: Status 403 while fetching image layer (91a6195f52a2c01646c3c5952c1fcc8e15f9d2a5ac049edb0ec8412a1cd1c6fe)\r\nJun 22 13:10:58 ip-172-31-36-121.ec2.internal sdnotify-proxy[4388]: 73b8daf50cec: Error pulling image (0.4.0) from quay.io/coreos/flannel, Server error: Status 403 while fetching image layer (91a6195f52a2c01646c3c5952c1fcc8e15f9d2a5ac049edb0ec8412a1cd1c6fe)\r\nJun 22 13:10:58 ip-172-31-36-121.ec2.internal dockerd[596]: Error pulling image (0.4.0) from quay.io/coreos/flannel, Server error: Status 403 while fetching image layer (91a6195f52a2c01646c3c5952c1fcc8e15f9d2a5ac049edb0ec8412a1cd1c6fe)\r\nJun 22 13:10:58 ip-172-31-36-121.ec2.internal dockerd[596]: time=\"2015-06-22T13:10:58Z\" level=info msg=\"-job pull(quay.io/coreos/flannel, 0.4.0) = ERR (1)\"\r\nJun 22 13:10:58 ip-172-31-36-121.ec2.internal sdnotify-proxy[4388]: time=\"2015-06-22T13:10:58Z\" level=fatal msg=\"Error pulling image (0.4.0) from quay.io/coreos/flannel, Server error: Status 403 while fetching image layer (91a6195f52a2c01646c3c5952c1fcc8e15f9d2a5ac049edb0ec8412a1cd1c6fe)\"\r\nJun 22 13:10:58 ip-172-31-36-121.ec2.internal systemd[1]: flanneld.service: main process exited, code=exited, status=1/FAILURE\r\nJun 22 13:10:58 ip-172-31-36-121.ec2.internal systemd[1]: Failed to start Network fabric for containers.\r\nJun 22 13:10:58 ip-172-31-36-121.ec2.internal systemd[1]: Unit flanneld.service entered failed state.\r\nJun 22 13:10:58 ip-172-31-36-121.ec2.internal systemd[1]: flanneld.service failed.\r\n```\r\n\r\nAny help will be much appreciated.\r\n\r\nThanks.\r\n"},"comment":{"url":"https://api.github.com/repos/coreos/flannel/issues/comments/114335516","html_url":"https://github.com/coreos/flannel/issues/218#issuecomment-114335516","issue_url":"https://api.github.com/repos/coreos/flannel/issues/218","id":114335516,"user":{"login":"misakwa","id":1142616,"avatar_url":"https://avatars.githubusercontent.com/u/1142616?v=3","gravatar_id":"","url":"https://api.github.com/users/misakwa","html_url":"https://github.com/misakwa","followers_url":"https://api.github.com/users/misakwa/followers","following_url":"https://api.github.com/users/misakwa/following{/other_user}","gists_url":"https://api.github.com/users/misakwa/gists{/gist_id}","starred_url":"https://api.github.com/users/misakwa/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/misakwa/subscriptions","organizations_url":"https://api.github.com/users/misakwa/orgs","repos_url":"https://api.github.com/users/misakwa/repos","events_url":"https://api.github.com/users/misakwa/events{/privacy}","received_events_url":"https://api.github.com/users/misakwa/received_events","type":"User","site_admin":false},"created_at":"2015-06-23T02:54:54Z","updated_at":"2015-06-23T02:54:54Z","body":"I really appreciate the prompt and consistent response from you all. Great work on the tools btw and thanks again."}}