{"action":"created","issue":{"url":"https://api.github.com/repos/weaveworks/weave/issues/1514","labels_url":"https://api.github.com/repos/weaveworks/weave/issues/1514/labels{/name}","comments_url":"https://api.github.com/repos/weaveworks/weave/issues/1514/comments","events_url":"https://api.github.com/repos/weaveworks/weave/issues/1514/events","html_url":"https://github.com/weaveworks/weave/issues/1514","id":110276599,"number":1514,"title":"Weave proxy unexpectedly closes the connection after /start request","user":{"login":"2opremio","id":2362916,"avatar_url":"https://avatars.githubusercontent.com/u/2362916?v=3","gravatar_id":"","url":"https://api.github.com/users/2opremio","html_url":"https://github.com/2opremio","followers_url":"https://api.github.com/users/2opremio/followers","following_url":"https://api.github.com/users/2opremio/following{/other_user}","gists_url":"https://api.github.com/users/2opremio/gists{/gist_id}","starred_url":"https://api.github.com/users/2opremio/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/2opremio/subscriptions","organizations_url":"https://api.github.com/users/2opremio/orgs","repos_url":"https://api.github.com/users/2opremio/repos","events_url":"https://api.github.com/users/2opremio/events{/privacy}","received_events_url":"https://api.github.com/users/2opremio/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/weaveworks/weave/labels/bug","name":"bug","color":"fc2929"},{"url":"https://api.github.com/repos/weaveworks/weave/labels/[component/proxy]","name":"[component/proxy]","color":"fef2c0"}],"state":"open","locked":false,"assignee":{"login":"paulbellamy","id":250199,"avatar_url":"https://avatars.githubusercontent.com/u/250199?v=3","gravatar_id":"","url":"https://api.github.com/users/paulbellamy","html_url":"https://github.com/paulbellamy","followers_url":"https://api.github.com/users/paulbellamy/followers","following_url":"https://api.github.com/users/paulbellamy/following{/other_user}","gists_url":"https://api.github.com/users/paulbellamy/gists{/gist_id}","starred_url":"https://api.github.com/users/paulbellamy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/paulbellamy/subscriptions","organizations_url":"https://api.github.com/users/paulbellamy/orgs","repos_url":"https://api.github.com/users/paulbellamy/repos","events_url":"https://api.github.com/users/paulbellamy/events{/privacy}","received_events_url":"https://api.github.com/users/paulbellamy/received_events","type":"User","site_admin":false},"milestone":{"url":"https://api.github.com/repos/weaveworks/weave/milestones/16","html_url":"https://github.com/weaveworks/weave/milestones/1.1.2","labels_url":"https://api.github.com/repos/weaveworks/weave/milestones/16/labels","id":1338814,"number":16,"title":"1.1.2","description":"","creator":{"login":"rade","id":109109,"avatar_url":"https://avatars.githubusercontent.com/u/109109?v=3","gravatar_id":"","url":"https://api.github.com/users/rade","html_url":"https://github.com/rade","followers_url":"https://api.github.com/users/rade/followers","following_url":"https://api.github.com/users/rade/following{/other_user}","gists_url":"https://api.github.com/users/rade/gists{/gist_id}","starred_url":"https://api.github.com/users/rade/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rade/subscriptions","organizations_url":"https://api.github.com/users/rade/orgs","repos_url":"https://api.github.com/users/rade/repos","events_url":"https://api.github.com/users/rade/events{/privacy}","received_events_url":"https://api.github.com/users/rade/received_events","type":"User","site_admin":false},"open_issues":3,"closed_issues":2,"state":"open","created_at":"2015-10-05T18:22:00Z","updated_at":"2015-10-12T13:58:09Z","due_on":null,"closed_at":null},"comments":17,"created_at":"2015-10-07T17:29:00Z","updated_at":"2015-10-14T09:55:11Z","closed_at":null,"body":"Very similar to #1103 but not a duplicate.  In an ECS-optimized AMI in Amazon's EC2, `amazon-ecs-init start` fails when using the weave proxy (DOCKER_HOST=unix:///var/run/weave.sock)\r\n\r\n```\r\n2015-10-07T17:03:30Z [INFO] pre-start\r\n2015-10-07T17:03:30Z [INFO] start\r\n2015-10-07T17:03:30Z [INFO] Container name: /weaveproxy\r\n2015-10-07T17:03:30Z [INFO] Container name: /weave\r\n2015-10-07T17:03:30Z [INFO] Container name: /ecs-weave-ecs-demo-task-438-httpserver-defec08acae0b2a32e00\r\n2015-10-07T17:03:30Z [INFO] Container name: /ecs-weave-ecs-demo-task-438-dataproducer-90f6e29c9a8fad9b1f00\r\n2015-10-07T17:03:30Z [INFO] Container name: /weavescope\r\n2015-10-07T17:03:30Z [INFO] No existing agent container to remove.\r\n2015-10-07T17:03:30Z [INFO] Starting Amazon EC2 Container Service Agent\r\n2015-10-07T17:03:31Z [ERROR] could not start Agent: persistent connection closed\r\n```\r\n\r\nThis seems to have been introduced in Weave 1.1.1 (I am not able to reproduce it with Weave 1.1).\r\n\r\nHere are the weaveproxy logs: https://gist.github.com/2opremio/59f5317189c26fb5fe67\r\n\r\nNote the `Error forwarding chunked response body: write unix @: broken pipe`\r\n\r\nHere is the communication between the `amazon-ecs-init`  (and `ecs-agent`) captured by `socat`: https://gist.github.com/2opremio/bc42d5d824e23d259c16\r\n\r\nFor future reference, here's how I sniffed the communication between `ecs-init` and the proxy:\r\n```\r\nsudo yum install -y socat\r\nsudo mv /var/run/weave.sock /var/run/weave.sock.orig\r\nsudo socat -t100  -v UNIX-LISTEN:/var/run/weave.sock,mode=777,reuseaddr,fork UNIX-CONNECT:/var/run/weave.sock.orig 2> /tmp/socat.out &\r\nsudo start ecs\r\n```\r\n\r\nAnd finally, here's the same communication, but using the docker daemon directly: https://gist.github.com/2opremio/6c7f066865fb9b0e38b7\r\n\r\nHere are some details about the docker version used:\r\n```\r\n$ docker version\r\nClient version: 1.7.1\r\nClient API version: 1.19\r\nGo version (client): go1.4.2\r\nGit commit (client): 786b29d/1.7.1\r\nOS/Arch (client): linux/amd64\r\nServer version: 1.7.1\r\nServer API version: 1.19\r\nGo version (server): go1.4.2\r\nGit commit (server): 786b29d/1.7.1\r\nOS/Arch (server): linux/amd64\r\n```\r\n\r\nNote how the Weave Proxy closes the connection after the `/start` request: https://gist.github.com/2opremio/bc42d5d824e23d259c16#file-socat-out-L123\r\n\r\nwhereas Docker doesn't: https://gist.github.com/2opremio/6c7f066865fb9b0e38b7#file-socat-docker-out-L119"},"comment":{"url":"https://api.github.com/repos/weaveworks/weave/issues/comments/147995061","html_url":"https://github.com/weaveworks/weave/issues/1514#issuecomment-147995061","issue_url":"https://api.github.com/repos/weaveworks/weave/issues/1514","id":147995061,"user":{"login":"paulbellamy","id":250199,"avatar_url":"https://avatars.githubusercontent.com/u/250199?v=3","gravatar_id":"","url":"https://api.github.com/users/paulbellamy","html_url":"https://github.com/paulbellamy","followers_url":"https://api.github.com/users/paulbellamy/followers","following_url":"https://api.github.com/users/paulbellamy/following{/other_user}","gists_url":"https://api.github.com/users/paulbellamy/gists{/gist_id}","starred_url":"https://api.github.com/users/paulbellamy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/paulbellamy/subscriptions","organizations_url":"https://api.github.com/users/paulbellamy/orgs","repos_url":"https://api.github.com/users/paulbellamy/repos","events_url":"https://api.github.com/users/paulbellamy/events{/privacy}","received_events_url":"https://api.github.com/users/paulbellamy/received_events","type":"User","site_admin":false},"created_at":"2015-10-14T09:55:11Z","updated_at":"2015-10-14T09:55:11Z","body":"It was fixed here: https://github.com/golang/go/blob/master/src/net/http/server.go#L917\r\n\r\nWe should close this as not_our_problem, as it's fixed in Go 1.5."}}