{"action":"created","issue":{"url":"https://api.github.com/repos/joyent/node/issues/14477","labels_url":"https://api.github.com/repos/joyent/node/issues/14477/labels{/name}","comments_url":"https://api.github.com/repos/joyent/node/issues/14477/comments","events_url":"https://api.github.com/repos/joyent/node/issues/14477/events","html_url":"https://github.com/joyent/node/issues/14477","id":66564403,"number":14477,"title":"When only active transform stream exceeds highWaterMark, process exits","user":{"login":"joepie91","id":1663259,"avatar_url":"https://avatars.githubusercontent.com/u/1663259?v=3","gravatar_id":"","url":"https://api.github.com/users/joepie91","html_url":"https://github.com/joepie91","followers_url":"https://api.github.com/users/joepie91/followers","following_url":"https://api.github.com/users/joepie91/following{/other_user}","gists_url":"https://api.github.com/users/joepie91/gists{/gist_id}","starred_url":"https://api.github.com/users/joepie91/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/joepie91/subscriptions","organizations_url":"https://api.github.com/users/joepie91/orgs","repos_url":"https://api.github.com/users/joepie91/repos","events_url":"https://api.github.com/users/joepie91/events{/privacy}","received_events_url":"https://api.github.com/users/joepie91/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2015-04-06T09:07:33Z","updated_at":"2015-04-10T19:37:09Z","closed_at":null,"body":"Node.js version: `v0.12.0`\r\n\r\nTest case:\r\n\r\n```javascript\r\nvar fs = require(\"fs\");\r\nvar stream = require(\"stream\");\r\n\r\nvar transform = new stream.Transform({\r\n  highWaterMark: 200 * 1024\r\n});\r\n\r\ntransform._transform = function(chunk, encoding, callback) {\r\n  console.log(\"Got chunk, length\", chunk.length);\r\n  this.push(chunk);\r\n  return callback();\r\n};\r\n\r\nconsole.log(\"Starting stream...\");\r\n\r\nfs.createReadStream(\"./lower.txt\")\r\n  .pipe(transform)\r\n  .on(\"end\", function() {\r\n      console.log(\"Done!\");\r\n  });\r\n```\r\n\r\nExpected result: Buffer fills up, process/stream stalls until buffer is empty enough again.\r\n\r\nActual result: Process exits without ever completing the stream, and without throwing any kind of error.\r\n\r\n```\r\nsven@linux-etoq:~/projects/node-bhttp> coffee test-exit.coffee \r\nStarting stream...\r\nGot chunk, length 65536\r\nGot chunk, length 65536\r\nGot chunk, length 65536\r\nGot chunk, length 65536\r\nsven@linux-etoq:~/projects/node-bhttp> \r\n```\r\n\r\nI presume that this is because the stream is paused and the event loop runs out of events, but it's very unintuitive, and it took me a good while to debug when I ran into it."},"comment":{"url":"https://api.github.com/repos/joyent/node/issues/comments/91661182","html_url":"https://github.com/joyent/node/issues/14477#issuecomment-91661182","issue_url":"https://api.github.com/repos/joyent/node/issues/14477","id":91661182,"user":{"login":"misterdjules","id":811047,"avatar_url":"https://avatars.githubusercontent.com/u/811047?v=3","gravatar_id":"","url":"https://api.github.com/users/misterdjules","html_url":"https://github.com/misterdjules","followers_url":"https://api.github.com/users/misterdjules/followers","following_url":"https://api.github.com/users/misterdjules/following{/other_user}","gists_url":"https://api.github.com/users/misterdjules/gists{/gist_id}","starred_url":"https://api.github.com/users/misterdjules/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/misterdjules/subscriptions","organizations_url":"https://api.github.com/users/misterdjules/orgs","repos_url":"https://api.github.com/users/misterdjules/repos","events_url":"https://api.github.com/users/misterdjules/events{/privacy}","received_events_url":"https://api.github.com/users/misterdjules/received_events","type":"User","site_admin":false},"created_at":"2015-04-10T19:37:09Z","updated_at":"2015-04-10T19:37:09Z","body":"@joepie91 @dayuoba My analysis and description of the problem was missing important details, thank you for pointing them out :+1:\r\n\r\n@joepie91 Indeed, the sample code I mentioned in [my previous comment](https://github.com/joyent/node/issues/14477#issuecomment-90307698) should not have set a listener for the `data` event for the transform stream.\r\n\r\nWith the sample code that uses the `net.Server` and `nc`, what happens if we don't trigger the flowing mode on the transform stream is that [`readStop` is called on the net.Socket readable stream](https://github.com/joyent/node/blob/master/lib/net.js#L542-L545), and that removes the underlying handle from the active handle at the libuv layer.\r\n\r\nIf there's nothing else that would keep the libuv loop open, then node exits. I think that's what you described in your EDIT2 @dayuoba. In the case of a network server, that use case would work because most of the time the server handle is kept active, and that at least keeps the libuv loop active.\r\n\r\nSo the sample code that uses a net.Server and a TCP client fails for similar reasons than the `fs.ReadStream` sample, but the technical details are slightly different.\r\n\r\nBasically, node streams do not hold the loop open, even if there's still data to process. I don't know if it's by design, and if so what was the rationale behind this decision. I haven't thought about the pros and cons of both approaches, but making streams hold a loop open or have an underlying handle would certainly not be something to consider lightly.\r\n\r\n@joyent/node-coreteam Thoughts about whether this is a bug, or if this works by design? Any recommendation for workarounds?"}}