{"action":"created","issue":{"url":"https://api.github.com/repos/docker/spdystream/issues/56","labels_url":"https://api.github.com/repos/docker/spdystream/issues/56/labels{/name}","comments_url":"https://api.github.com/repos/docker/spdystream/issues/56/comments","events_url":"https://api.github.com/repos/docker/spdystream/issues/56/events","html_url":"https://github.com/docker/spdystream/pull/56","id":104539931,"number":56,"title":"Fix possible data loss after goaway received","user":{"login":"ncdc","id":354,"avatar_url":"https://avatars.githubusercontent.com/u/354?v=3","gravatar_id":"","url":"https://api.github.com/users/ncdc","html_url":"https://github.com/ncdc","followers_url":"https://api.github.com/users/ncdc/followers","following_url":"https://api.github.com/users/ncdc/following{/other_user}","gists_url":"https://api.github.com/users/ncdc/gists{/gist_id}","starred_url":"https://api.github.com/users/ncdc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ncdc/subscriptions","organizations_url":"https://api.github.com/users/ncdc/orgs","repos_url":"https://api.github.com/users/ncdc/repos","events_url":"https://api.github.com/users/ncdc/events{/privacy}","received_events_url":"https://api.github.com/users/ncdc/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":10,"created_at":"2015-09-02T18:04:25Z","updated_at":"2015-10-16T20:03:42Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/docker/spdystream/pulls/56","html_url":"https://github.com/docker/spdystream/pull/56","diff_url":"https://github.com/docker/spdystream/pull/56.diff","patch_url":"https://github.com/docker/spdystream/pull/56.patch"},"body":"All frames are processed by workers running in goroutines. If 1 worker's\r\nqueue is backed up with data frames and another worker receives and\r\nprocesses a goaway frame, it's a race as to whether or not the workers\r\nwill be able to process the pending data frames before the streams are\r\nremoved from the streams map.\r\n\r\nThis fix processes the goaway frame synchronously in Serve() instead of\r\nin a worker goroutine. We now wait for all the workers to drain their\r\nframe queues before tearing down the streams.\r\n\r\nFixes #55"},"comment":{"url":"https://api.github.com/repos/docker/spdystream/issues/comments/148820584","html_url":"https://github.com/docker/spdystream/pull/56#issuecomment-148820584","issue_url":"https://api.github.com/repos/docker/spdystream/issues/56","id":148820584,"user":{"login":"ncdc","id":354,"avatar_url":"https://avatars.githubusercontent.com/u/354?v=3","gravatar_id":"","url":"https://api.github.com/users/ncdc","html_url":"https://github.com/ncdc","followers_url":"https://api.github.com/users/ncdc/followers","following_url":"https://api.github.com/users/ncdc/following{/other_user}","gists_url":"https://api.github.com/users/ncdc/gists{/gist_id}","starred_url":"https://api.github.com/users/ncdc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ncdc/subscriptions","organizations_url":"https://api.github.com/users/ncdc/orgs","repos_url":"https://api.github.com/users/ncdc/repos","events_url":"https://api.github.com/users/ncdc/events{/privacy}","received_events_url":"https://api.github.com/users/ncdc/received_events","type":"User","site_admin":false},"created_at":"2015-10-16T20:03:42Z","updated_at":"2015-10-16T20:03:42Z","body":"Coming back around to this today. I still haven't built a reliable reproducer without forcibly injecting some delays into the main code, but I was able to do so with that approach. I modified the frame handler so that it sleeps 1 second before processing any data frame. In the test case, have a client send multiple data frames over 1 single stream to the server. Have the server verify it receives everything the client sends. Without this fix, you'll see multiple \"Data frame gone away for 1\" messages as the server tries to process pending data frames after it's already processed the go away frame. With the fix, all frames are delivered, and the test passes. So the only way I can think of right now to reliably test this would be to add some code that evaluates an internal `bool` set by the test code that forces delayed delivery of data frames. I'm not sure that's a good idea, but it's the best thing I have so far...\r\n\r\n@dmcgowan how do you want to proceed?"}}