{"action":"created","issue":{"url":"https://api.github.com/repos/nodejs/node/issues/2485","labels_url":"https://api.github.com/repos/nodejs/node/issues/2485/labels{/name}","comments_url":"https://api.github.com/repos/nodejs/node/issues/2485/comments","events_url":"https://api.github.com/repos/nodejs/node/issues/2485/events","html_url":"https://github.com/nodejs/node/issues/2485","id":102364141,"number":2485,"title":"zlib streams emitting data after end","user":{"login":"jhamhader","id":7780985,"avatar_url":"https://avatars.githubusercontent.com/u/7780985?v=3","gravatar_id":"","url":"https://api.github.com/users/jhamhader","html_url":"https://github.com/jhamhader","followers_url":"https://api.github.com/users/jhamhader/followers","following_url":"https://api.github.com/users/jhamhader/following{/other_user}","gists_url":"https://api.github.com/users/jhamhader/gists{/gist_id}","starred_url":"https://api.github.com/users/jhamhader/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jhamhader/subscriptions","organizations_url":"https://api.github.com/users/jhamhader/orgs","repos_url":"https://api.github.com/users/jhamhader/repos","events_url":"https://api.github.com/users/jhamhader/events{/privacy}","received_events_url":"https://api.github.com/users/jhamhader/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/nodejs/node/labels/zlib","name":"zlib","color":"009800"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2015-08-21T12:05:29Z","updated_at":"2015-08-24T17:38:51Z","closed_at":null,"body":"Hey,\r\n\r\nI noticed that zlib streams still emit data (i.e in deflate stream, it writes the adler32 checksum) after .end() has been called.\r\nHere is a test (with useful prints, obviously should be removed later) which fails on the recent 3.0.x:\r\n(notice the  data written after \"deflate end cb\")\r\n\r\n```\r\nvar zlib = require('zlib');\r\n\r\nvar input = 'Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Aenean commodo ligula eget dolor. Aenean massa.'\r\nvar deflate = zlib.createDeflate();\r\nvar alreadyEnded = 0;\r\n\r\ndeflate.on('data', function(chunk) {\r\n        console.log('deflate data:');\r\n        console.log(chunk);\r\n        if (alreadyEnded) {\r\n                throw \"Stream already ended\"\r\n        }   \r\n});\r\n\r\ndeflate.on('prefinish', function() {\r\n        console.log('deflate prefinish event');\r\n});\r\n\r\ndeflate.on('finish', function() {\r\n        console.log('deflate finish event');\r\n});\r\n\r\ndeflate.write(input);\r\ndefalte.flush(function() {\r\n        deflate.end(function () {\r\n                console.log('deflate end cb');\r\n                alreadyEnded = 1;\r\n        });\r\n});\r\n```\r\nI would really like to help fix this issue and it to be my first contribution here, so if anyone is willing to help with mentoring, that will be great.\r\nFrom what I understand, calling end() triggers calling endWritable() which calls prefinish().\r\nTransform's _flush() is registered to the prefinish event. Problem is, _flush() calls an **async** zlib Write(). Meanwhile, endWritable continues and calls end()'s callback.\r\n\r\nAm I on to an issue?\r\nShould zlib's _flush write synchronously if this is the last write?"},"comment":{"url":"https://api.github.com/repos/nodejs/node/issues/comments/134312232","html_url":"https://github.com/nodejs/node/issues/2485#issuecomment-134312232","issue_url":"https://api.github.com/repos/nodejs/node/issues/2485","id":134312232,"user":{"login":"chrisdickinson","id":37303,"avatar_url":"https://avatars.githubusercontent.com/u/37303?v=3","gravatar_id":"","url":"https://api.github.com/users/chrisdickinson","html_url":"https://github.com/chrisdickinson","followers_url":"https://api.github.com/users/chrisdickinson/followers","following_url":"https://api.github.com/users/chrisdickinson/following{/other_user}","gists_url":"https://api.github.com/users/chrisdickinson/gists{/gist_id}","starred_url":"https://api.github.com/users/chrisdickinson/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/chrisdickinson/subscriptions","organizations_url":"https://api.github.com/users/chrisdickinson/orgs","repos_url":"https://api.github.com/users/chrisdickinson/repos","events_url":"https://api.github.com/users/chrisdickinson/events{/privacy}","received_events_url":"https://api.github.com/users/chrisdickinson/received_events","type":"User","site_admin":false},"created_at":"2015-08-24T17:38:51Z","updated_at":"2015-08-24T17:38:51Z","body":"It's a little confusing, but [`end()`](https://github.com/nodejs/node/blob/2d251e661a8c761e0c1bd809fdc6a29179950c00/lib/_stream_writable.js#L420-L444) doesn't [directly](https://github.com/nodejs/node/blob/2d251e661a8c761e0c1bd809fdc6a29179950c00/lib/_stream_writable.js#L476-L486) correlate to the `'end'` event in transform streams. Instead, it corresponds to [`'finish'` on the writable side of the stream](https://github.com/nodejs/node/blob/2d251e661a8c761e0c1bd809fdc6a29179950c00/lib/_stream_writable.js#L462-L474). That is, once the data has been *written* and *processed by* to the stream's backing resource, it will emit `\"prefinish\"`, then `\"finish\"`. `\"prefinish\"`, which most interests us at this point, is fired the moment we get the `end` call. `\"prefinish\"` triggers the [`_flush`](https://github.com/nodejs/node/blob/2d251e661a8c761e0c1bd809fdc6a29179950c00/lib/_stream_transform.js#L116-L123) machinery in Transform streams, which in turn, triggers zlib's [`_flush`](https://github.com/nodejs/node/blob/2d251e661a8c761e0c1bd809fdc6a29179950c00/lib/zlib.js#L426-L430), which will force zlib to spit out the crc trailer as a `\"data\"` event and call the callback, which calls code in Transform that [ends the readable side of the stream](https://github.com/nodejs/node/blob/2d251e661a8c761e0c1bd809fdc6a29179950c00/lib/_stream_transform.js#L176-L192). Note: the \"end\" event won't be fired until the data is pulled _out_ of the readable side of the buffer — in this case, the `.on('data')` listener is doing the pulling — so we end up with:\r\n\r\n```\r\n1. \"prefinish\"\r\n  1. _flush                              1. <writable cbs are exhausted>\r\n  2. zlib._flush                         2. \"finish\"\r\n  3. zlib pushes empty buffer            3. user-supplied end() cb called\r\n  4. <time passes>\r\n  5. zlib emits trailer info\r\n  6. zlib ends stream with `push(null)`\r\n  7. stream emits \"data\"\r\n  8. stream emits \"end\"\r\n```\r\n\r\nData isn't appearing after the `\"end\"` event, it's appearing between `\"finish\"` and `\"end\"` — which is an okay thing for data to do. It is pretty confusing that `end()`'s callback fires on `\"finish\"`, though.\r\n\r\nHopefully this clarifies what's going on in the above example!\r\n\r\n(I've omitted the manual `.flush` call, since that's not part of the streams spec, and wouldn't affect the outcome.)\r\n"}}