{"action":"opened","issue":{"url":"https://api.github.com/repos/netty/netty/issues/4242","labels_url":"https://api.github.com/repos/netty/netty/issues/4242/labels{/name}","comments_url":"https://api.github.com/repos/netty/netty/issues/4242/comments","events_url":"https://api.github.com/repos/netty/netty/issues/4242/events","html_url":"https://github.com/netty/netty/issues/4242","id":107389418,"number":4242,"title":"HTTP/2 FlowController allocated write and flush","user":{"login":"blucas","id":1743698,"avatar_url":"https://avatars.githubusercontent.com/u/1743698?v=3","gravatar_id":"","url":"https://api.github.com/users/blucas","html_url":"https://github.com/blucas","followers_url":"https://api.github.com/users/blucas/followers","following_url":"https://api.github.com/users/blucas/following{/other_user}","gists_url":"https://api.github.com/users/blucas/gists{/gist_id}","starred_url":"https://api.github.com/users/blucas/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/blucas/subscriptions","organizations_url":"https://api.github.com/users/blucas/orgs","repos_url":"https://api.github.com/users/blucas/repos","events_url":"https://api.github.com/users/blucas/events{/privacy}","received_events_url":"https://api.github.com/users/blucas/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2015-09-20T11:45:43Z","updated_at":"2015-09-20T11:45:43Z","closed_at":null,"body":"Netty Version: lastest master (94bf412edbea73c0b1dbac4ada8919708c4f369e)\r\n\r\n/cc @Scottmitch @nmittler \r\n\r\nI have a serious issue (it is actually a blocker) that is preventing me from releasing the next version of our product (which will support HTTP/2 traffic).  I would really appreciate it if this could be resolved quickly.  I am more than willing to help out with this (submit a PR) as long as someone can help point me in the right direction to fixing this issue.\r\n\r\n**Issue Details**\r\nWhen sending a large response to a client, the flow controller fails to send the entire response.  This is very similar to #4052, but is not solved by the fix for that issue. \r\n\r\nThe problem stems from the fact that the flow controller and state only write up to N allocated bytes [here](https://github.com/netty/netty/blob/master/codec-http2/src/main/java/io/netty/handler/codec/http2/DefaultHttp2RemoteFlowController.java#L544).  If there are still pending bytes to write, the flow controller should trigger another write, after flushing.  Unfortunately, at this time, all it will do is flush the N allocated bytes, something must be done so that it will trigger another write (and subsequent flush).  I don't know how it should accomplish that.  I am aware there is a pending pull request for 'stream writability' (#4227), but I don't believe it goes far enough to fix this issue on its own.\r\n\r\n**Example**\r\nGiven:\r\nHttpServer which receives a request, generates a ```FullHttpResponse```, and calls ```writeAndFlush```. The pipeline is configured to convert the FullHttpResponse to HTTP/2 Frames.\r\n\r\nResponse / Channel / Stream Info:\r\nResponse size: 100,000 bytes\r\nChannel WriteBuffer High Watermark: 65,535 bytes\r\nStream allocated bytes: 43,000 bytes\r\n\r\nGiven the scenario above:\r\n\r\n1. ```Http2ConnectionHandler``` receives a call to [flush](https://github.com/netty/netty/blob/master/codec-http2/src/main/java/io/netty/handler/codec/http2/Http2ConnectionHandler.java#L209) the converted ```FullHttpResponse``` and tells the flow controller to write the pending bytes.\r\n2. the flow controller calculates the number of bytes to write ([allocate](https://github.com/netty/netty/blob/master/codec-http2/src/main/java/io/netty/handler/codec/http2/DefaultHttp2RemoteFlowController.java#L305)) for the stream.\r\n3. the flow controller then triggers a [write N allocated bytes](https://github.com/netty/netty/blob/master/codec-http2/src/main/java/io/netty/handler/codec/http2/DefaultHttp2RemoteFlowController.java#L310) for each stream.\r\n4. DefaultState will [write](https://github.com/netty/netty/blob/master/codec-http2/src/main/java/io/netty/handler/codec/http2/DefaultHttp2RemoteFlowController.java#L544) 43,000 bytes down the pipeline.\r\n5. ```Http2ConnectionHandler``` will then trigger a [flush](https://github.com/netty/netty/blob/master/codec-http2/src/main/java/io/netty/handler/codec/http2/Http2ConnectionHandler.java#L211) to flush the 43,000 bytes to the socket.\r\n\r\nSo the problem is that only the first 43,000 bytes were sent to the socket. The other 57,000 bytes still need to be sent.\r\n\r\nThe reason we did not get this problem with #4052 was because the ```Stream allocate bytes``` was set to a value much closer to the ```Channel WriteBuffer High Watermark```. That triggered a ```channelWritabilityChanged``` event, which ended up writing/flushing more and more of the response to the socket.  Below is an example of what I mean:\r\n\r\nResponse / Channel / Stream Info:\r\nResponse size: 100,000 bytes\r\nChannel WriteBuffer High Watermark: 65,535 bytes\r\nStream allocated bytes: 65,500 bytes\r\n\r\nGiven the scenario above:\r\n\r\n1. ```Http2ConnectionHandler``` receives a call to [flush](https://github.com/netty/netty/blob/master/codec-http2/src/main/java/io/netty/handler/codec/http2/Http2ConnectionHandler.java#L209) the converted ```FullHttpResponse``` and tells the flow controller to write the pending bytes.\r\n2. the flow controller calculates the number of bytes to write ([allocate](https://github.com/netty/netty/blob/master/codec-http2/src/main/java/io/netty/handler/codec/http2/DefaultHttp2RemoteFlowController.java#L305)) for the stream.\r\n3. the flow controller then triggers a [write N allocated bytes](https://github.com/netty/netty/blob/master/codec-http2/src/main/java/io/netty/handler/codec/http2/DefaultHttp2RemoteFlowController.java#L310) for each stream.\r\n4. DefaultState will [write](https://github.com/netty/netty/blob/master/codec-http2/src/main/java/io/netty/handler/codec/http2/DefaultHttp2RemoteFlowController.java#L544) 65,500 bytes down the pipeline.\r\n5. ```Http2ConnectionHandler``` will then trigger a [flush](https://github.com/netty/netty/blob/master/codec-http2/src/main/java/io/netty/handler/codec/http2/Http2ConnectionHandler.java#L211) to flush the 65,500 bytes to the socket.\r\n6. ```SslHandler``` encrypts the 65,500 bytes, adding a further N number of bytes thus triggering a ```channelWritabilityChanged``` event.\r\n7. [ChannelWritabilityChanged](https://github.com/netty/netty/blob/master/codec-http2/src/main/java/io/netty/handler/codec/http2/Http2ConnectionHandler.java#L441) event triggers further writes/flushes to the socket.\r\n\r\nA side note to all of this... I spotted something that might be a bug, or maybe it is just incorrect documentation.  The code [here](https://github.com/netty/netty/blob/master/codec-http2/src/main/java/io/netty/handler/codec/http2/DefaultHttp2RemoteFlowController.java#L652) states that all the frame data was written, but the ```if``` is checking ```frame.size() != 0```.  Should it actually be ```frame.size() == 0``` ???  Or maybe the documentation needs clearing up?\r\n\r\n**Reproducer**\r\n@Scottmitch - The reproducer I gave you for #4052 can be used to produce the scenario I've stated above.  In order to reproduce the issue, just request the file using Firefox (version I used was v40.0.3) instead of Chrome.  The HTTP/2 codec sets the stream's allocated bytes much lower for Firefox than it does from Chrome. "}}