{"action":"closed","issue":{"url":"https://api.github.com/repos/coreos/etcd/issues/3118","labels_url":"https://api.github.com/repos/coreos/etcd/issues/3118/labels{/name}","comments_url":"https://api.github.com/repos/coreos/etcd/issues/3118/comments","events_url":"https://api.github.com/repos/coreos/etcd/issues/3118/events","html_url":"https://github.com/coreos/etcd/issues/3118","id":94648173,"number":3118,"title":"etcdserver: publish error: etcdserver: request timed out","user":{"login":"Pensu","id":4402424,"avatar_url":"https://avatars.githubusercontent.com/u/4402424?v=3","gravatar_id":"","url":"https://api.github.com/users/Pensu","html_url":"https://github.com/Pensu","followers_url":"https://api.github.com/users/Pensu/followers","following_url":"https://api.github.com/users/Pensu/following{/other_user}","gists_url":"https://api.github.com/users/Pensu/gists{/gist_id}","starred_url":"https://api.github.com/users/Pensu/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Pensu/subscriptions","organizations_url":"https://api.github.com/users/Pensu/orgs","repos_url":"https://api.github.com/users/Pensu/repos","events_url":"https://api.github.com/users/Pensu/events{/privacy}","received_events_url":"https://api.github.com/users/Pensu/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":9,"created_at":"2015-07-13T06:17:27Z","updated_at":"2015-07-15T20:06:36Z","closed_at":"2015-07-15T20:06:36Z","body":"Hi all,\r\n\r\nI am running two VMs on a single host and I am trying to create an etcd cluster using them. Here are the results of running etcd on both:\r\n\r\nVM 1:\r\n\r\n```\r\n ./bin/etcd -name infra0 -initial-advertise-peer-urls http://192.168.122.42:2380 -listen-peer-urls http://192.168.122.42:2380 -listen-client-urls http://192.168.122.42:2379,http://127.0.0.1:2379 -advertise-client-urls http://192.168.122.42:2379 -initial-cluster-token etcd-cluster-1 -initial-cluster infra0=http://192.168.122.42:2380,infra1=http://192.168.122.119:2380 -initial-cluster-state new\r\n2015/07/13 01:13:06 etcdmain: setting maximum number of CPUs to 1, total number of available CPUs is 1\r\n2015/07/13 01:13:06 etcdmain: no data-dir provided, using default data-dir ./infra0.etcd\r\n2015/07/13 01:13:06 etcdmain: the server is already initialized as member before, starting as etcd member...\r\n2015/07/13 01:13:06 etcdmain: listening for peers on http://192.168.122.42:2380\r\n2015/07/13 01:13:06 etcdmain: listening for client requests on http://127.0.0.1:2379\r\n2015/07/13 01:13:06 etcdmain: listening for client requests on http://192.168.122.42:2379\r\n2015/07/13 01:13:06 etcdserver: name = infra0\r\n2015/07/13 01:13:06 etcdserver: data dir = infra0.etcd\r\n2015/07/13 01:13:06 etcdserver: member dir = infra0.etcd/member\r\n2015/07/13 01:13:06 etcdserver: heartbeat = 100ms\r\n2015/07/13 01:13:06 etcdserver: election = 1000ms\r\n2015/07/13 01:13:06 etcdserver: snapshot count = 10000\r\n2015/07/13 01:13:06 etcdserver: advertise client URLs = http://192.168.122.42:2379\r\n2015/07/13 01:13:06 etcdserver: restarting member ca65234765552f1a in cluster ed2bdaf962eada24 at commit index 2\r\n2015/07/13 01:13:06 raft: ca65234765552f1a became follower at term 19\r\n2015/07/13 01:13:06 raft: newRaft ca65234765552f1a [peers: [], term: 19, commit: 2, applied: 0, lastindex: 2, lastterm: 1]\r\n2015/07/13 01:13:06 etcdserver: starting server... [version: 2.1.0-rc.0+git, cluster version: to_be_decided]\r\n2015/07/13 01:13:06 etcdserver: added member 15fc93be0da97fd4 [http://192.168.122.119:2380] to cluster ed2bdaf962eada24\r\n2015/07/13 01:13:06 etcdserver: added local member ca65234765552f1a [http://192.168.122.42:2380] to cluster ed2bdaf962eada24\r\n2015/07/13 01:13:06 rafthttp: failed to dial 15fc93be0da97fd4 on stream Message (dial tcp 192.168.122.119:2380: connection refused)\r\n2015/07/13 01:13:06 rafthttp: failed to dial 15fc93be0da97fd4 on stream MsgApp v2 (dial tcp 192.168.122.119:2380: connection refused)\r\n2015/07/13 01:13:07 raft: ca65234765552f1a is starting a new election at term 19\r\n2015/07/13 01:13:07 raft: ca65234765552f1a became candidate at term 20\r\n2015/07/13 01:13:07 raft: ca65234765552f1a received vote from ca65234765552f1a at term 20\r\n2015/07/13 01:13:07 raft: ca65234765552f1a [logterm: 1, index: 2] sent vote request to 15fc93be0da97fd4 at term 20\r\n2015/07/13 01:13:07 rafthttp: failed to write 15fc93be0da97fd4 on pipeline (dial tcp 192.168.122.119:2380: connection refused)\r\n2015/07/13 01:13:08 rafthttp: the connection with 15fc93be0da97fd4 became active\r\n2015/07/13 01:13:09 raft: ca65234765552f1a is starting a new election at term 20\r\n2015/07/13 01:13:09 raft: ca65234765552f1a became candidate at term 21\r\n2015/07/13 01:13:09 raft: ca65234765552f1a received vote from ca65234765552f1a at term 21\r\n2015/07/13 01:13:09 raft: ca65234765552f1a [logterm: 1, index: 2] sent vote request to 15fc93be0da97fd4 at term 21\r\n2015/07/13 01:13:10 raft: ca65234765552f1a is starting a new election at term 21\r\n2015/07/13 01:13:10 raft: ca65234765552f1a became candidate at term 22\r\n2015/07/13 01:13:10 raft: ca65234765552f1a received vote from ca65234765552f1a at term 22\r\n2015/07/13 01:13:10 raft: ca65234765552f1a [logterm: 1, index: 2] sent vote request to 15fc93be0da97fd4 at term 22\r\n2015/07/13 01:13:11 etcdserver: publish error: etcdserver: request timed out\r\n2015/07/13 01:13:11 raft: ca65234765552f1a is starting a new election at term 22\r\n2015/07/13 01:13:11 raft: ca65234765552f1a became candidate at term 23\r\n2015/07/13 01:13:11 raft: ca65234765552f1a received vote from ca65234765552f1a at term 23\r\n2015/07/13 01:13:11 raft: ca65234765552f1a [logterm: 1, index: 2] sent vote request to 15fc93be0da97fd4 at term 23\r\n2015/07/13 01:13:13 raft: ca65234765552f1a is starting a new election at term 23\r\n2015/07/13 01:13:13 raft: ca65234765552f1a became candidate at term 24\r\n2015/07/13 01:13:13 raft: ca65234765552f1a received vote from ca65234765552f1a at term 24\r\n2015/07/13 01:13:13 raft: ca65234765552f1a [logterm: 1, index: 2] sent vote request to 15fc93be0da97fd4 at term 24\r\n```\r\n\r\nVM 2:\r\n\r\n```\r\n./bin/etcd -name infra1 -initial-advertise-peer-urls http://192.168.122.119:2380 -listen-peer-urls http://192.168.122.119:2380 -listen-client-urls http://192.168.122.119:2379,http://127.0.0.1:2379 -advertise-client-urls http://192.168.122.119:2379 -initial-cluster-token etcd-cluster-1 -initial-cluster infra0=http://192.168.122.42:2380,infra1=http://192.168.122.119:2380 -initial-cluster-state new\r\n2015/07/13 01:13:08 etcdmain: setting maximum number of CPUs to 1, total number of available CPUs is 1\r\n2015/07/13 01:13:08 etcdmain: no data-dir provided, using default data-dir ./infra1.etcd\r\n2015/07/13 01:13:08 etcdmain: the server is already initialized as member before, starting as etcd member...\r\n2015/07/13 01:13:08 etcdmain: listening for peers on http://192.168.122.119:2380\r\n2015/07/13 01:13:08 etcdmain: listening for client requests on http://127.0.0.1:2379\r\n2015/07/13 01:13:08 etcdmain: listening for client requests on http://192.168.122.119:2379\r\n2015/07/13 01:13:08 etcdserver: name = infra1\r\n2015/07/13 01:13:08 etcdserver: data dir = infra1.etcd\r\n2015/07/13 01:13:08 etcdserver: member dir = infra1.etcd/member\r\n2015/07/13 01:13:08 etcdserver: heartbeat = 100ms\r\n2015/07/13 01:13:08 etcdserver: election = 1000ms\r\n2015/07/13 01:13:08 etcdserver: snapshot count = 10000\r\n2015/07/13 01:13:08 etcdserver: advertise client URLs = http://192.168.122.119:2379\r\n2015/07/13 01:13:08 etcdserver: restarting member 15fc93be0da97fd4 in cluster ed2bdaf962eada24 at commit index 2\r\n2015/07/13 01:13:08 raft: 15fc93be0da97fd4 became follower at term 17\r\n2015/07/13 01:13:08 raft: newRaft 15fc93be0da97fd4 [peers: [], term: 17, commit: 2, applied: 0, lastindex: 2, lastterm: 1]\r\n2015/07/13 01:13:08 etcdserver: starting server... [version: 2.1.0-rc.0+git, cluster version: to_be_decided]\r\n2015/07/13 01:13:08 etcdserver: added local member 15fc93be0da97fd4 [http://192.168.122.119:2380] to cluster ed2bdaf962eada24\r\n2015/07/13 01:13:08 etcdserver: added member ca65234765552f1a [http://192.168.122.42:2380] to cluster ed2bdaf962eada24\r\n2015/07/13 01:13:08 rafthttp: the connection with ca65234765552f1a became active\r\n2015/07/13 01:13:09 raft: 15fc93be0da97fd4 is starting a new election at term 17\r\n2015/07/13 01:13:09 raft: 15fc93be0da97fd4 became candidate at term 18\r\n2015/07/13 01:13:09 raft: 15fc93be0da97fd4 received vote from 15fc93be0da97fd4 at term 18\r\n2015/07/13 01:13:09 raft: 15fc93be0da97fd4 [logterm: 1, index: 2] sent vote request to ca65234765552f1a at term 18\r\n2015/07/13 01:13:11 raft: 15fc93be0da97fd4 is starting a new election at term 18\r\n2015/07/13 01:13:11 raft: 15fc93be0da97fd4 became candidate at term 19\r\n2015/07/13 01:13:11 raft: 15fc93be0da97fd4 received vote from 15fc93be0da97fd4 at term 19\r\n2015/07/13 01:13:11 raft: 15fc93be0da97fd4 [logterm: 1, index: 2] sent vote request to ca65234765552f1a at term 19\r\n2015/07/13 01:13:12 raft: 15fc93be0da97fd4 is starting a new election at term 19\r\n2015/07/13 01:13:12 raft: 15fc93be0da97fd4 became candidate at term 20\r\n2015/07/13 01:13:12 raft: 15fc93be0da97fd4 received vote from 15fc93be0da97fd4 at term 20\r\n2015/07/13 01:13:12 raft: 15fc93be0da97fd4 [logterm: 1, index: 2] sent vote request to ca65234765552f1a at term 20\r\n2015/07/13 01:13:13 etcdserver: publish error: etcdserver: request timed out\r\n2015/07/13 01:13:13 raft: 15fc93be0da97fd4 is starting a new election at term 20\r\n2015/07/13 01:13:13 raft: 15fc93be0da97fd4 became candidate at term 21\r\n2015/07/13 01:13:13 raft: 15fc93be0da97fd4 received vote from 15fc93be0da97fd4 at term 21\r\n2015/07/13 01:13:13 raft: 15fc93be0da97fd4 [logterm: 1, index: 2] sent vote request to ca65234765552f1a at term 21\r\n2015/07/13 01:13:14 rafthttp: the connection with ca65234765552f1a became inactive\r\n2015/07/13 01:13:14 rafthttp: failed to dial ca65234765552f1a on stream MsgApp v2 (dial tcp 192.168.122.42:2380: connection refused)\r\n2015/07/13 01:13:14 rafthttp: failed to dial ca65234765552f1a on stream Message (dial tcp 192.168.122.42:2380: connection refused)\r\n2015/07/13 01:13:15 raft: 15fc93be0da97fd4 is starting a new election at term 21\r\n2015/07/13 01:13:15 raft: 15fc93be0da97fd4 became candidate at term 22\r\n2015/07/13 01:13:15 raft: 15fc93be0da97fd4 received vote from 15fc93be0da97fd4 at term 22\r\n2015/07/13 01:13:15 raft: 15fc93be0da97fd4 [logterm: 1, index: 2] sent vote request to ca65234765552f1a at term 22\r\n2015/07/13 01:13:16 raft: 15fc93be0da97fd4 is starting a new election at term 22\r\n2015/07/13 01:13:16 raft: 15fc93be0da97fd4 became candidate at term 23\r\n2015/07/13 01:13:16 raft: 15fc93be0da97fd4 received vote from 15fc93be0da97fd4 at term 23\r\n2015/07/13 01:13:16 raft: 15fc93be0da97fd4 [logterm: 1, index: 2] sent vote request to ca65234765552f1a at term 23\r\n2015/07/13 01:13:16 rafthttp: failed to heartbeat ca65234765552f1a on stream Message (write tcp 192.168.122.42:51683: broken pipe)\r\n```\r\n\r\nAnd it keeps going on like this. I tried to check cluster health:\r\n\r\n```\r\n ./bin/etcdctl cluster-health\r\nError:  cannot sync with the cluster using endpoints http://127.0.0.1:4001, http://127.0.0.1:2379\r\n```\r\n\r\nThough I can connect to 127.0.0.1:2379 using curl:\r\n```\r\n$ curl http://127.0.0.1:2379/v2/keys/\r\n{\"action\":\"get\",\"node\":{\"dir\":true}}\r\n```\r\nSame happens when I am using discovery to create the cluster. Can you please help me figure this out? TIA."}}