{"action":"created","issue":{"url":"https://api.github.com/repos/brianc/node-pg-query-stream/issues/15","labels_url":"https://api.github.com/repos/brianc/node-pg-query-stream/issues/15/labels{/name}","comments_url":"https://api.github.com/repos/brianc/node-pg-query-stream/issues/15/comments","events_url":"https://api.github.com/repos/brianc/node-pg-query-stream/issues/15/events","html_url":"https://github.com/brianc/node-pg-query-stream/pull/15","id":115368800,"number":15,"title":"Avoid race when stream closed while fetching","user":{"login":"slickmb","id":3010941,"avatar_url":"https://avatars.githubusercontent.com/u/3010941?v=3","gravatar_id":"","url":"https://api.github.com/users/slickmb","html_url":"https://github.com/slickmb","followers_url":"https://api.github.com/users/slickmb/followers","following_url":"https://api.github.com/users/slickmb/following{/other_user}","gists_url":"https://api.github.com/users/slickmb/gists{/gist_id}","starred_url":"https://api.github.com/users/slickmb/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/slickmb/subscriptions","organizations_url":"https://api.github.com/users/slickmb/orgs","repos_url":"https://api.github.com/users/slickmb/repos","events_url":"https://api.github.com/users/slickmb/events{/privacy}","received_events_url":"https://api.github.com/users/slickmb/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2015-11-05T20:36:45Z","updated_at":"2015-11-13T16:37:27Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/brianc/node-pg-query-stream/pulls/15","html_url":"https://github.com/brianc/node-pg-query-stream/pull/15","diff_url":"https://github.com/brianc/node-pg-query-stream/pull/15.diff","patch_url":"https://github.com/brianc/node-pg-query-stream/pull/15.patch"},"body":"While working with node-pg-query-stream, I ran into an issue where the pg query stream was emitting large numbers of error events after I attempted to short-circuit the stream.\r\n\r\n```JavaScript\r\n\r\n\r\npg.connect(function (err, client, cb) {\r\n\r\n    var stream = new QueryStream('SELECT * FROM generate_series(0, 2000) num');\r\n    var query = client.query(stream);\r\n\r\n    var Writable = require('stream').Writable;\r\n    var items = 0;\r\n\r\n    var output = new Writable({ objectMode: true });\r\n    output._write = function (json, encoding, cb) {\r\n        process.nextTick(function () {\r\n            cb(items++); // first will pass, second will fail.\r\n        });\r\n    };\r\n\r\n    query.on('error', function (err) { console.error('ouch!', err); }); // will get thrown many time with \"Error: stream.push() after EOF\"\r\n    output.on('error', function () {\r\n\r\n        // downstream error.  let pg know we're done with the cursor before returning the client.\r\n        query.close();\r\n        cb();\r\n    });\r\n\r\n    output.on('finish', cb);\r\n});\r\n\r\n/* occasional console output:\r\n ouch! Error: stream.push() after EOF\r\n ouch! Error: stream.push() after EOF\r\n ouch! Error: stream.push() after EOF\r\n ouch! Error: stream.push() after EOF\r\n ouch! Error: stream.push() after EOF\r\n ouch! Error: stream.push() after EOF\r\n ouch! Error: stream.push() after EOF\r\n ouch! Error: stream.push() after EOF\r\n ouch! Error: stream.push() after EOF\r\n ouch! Error: stream.push() after EOF\r\n ouch! Error: stream.push() after EOF\r\n ouch! Error: stream.push() after EOF\r\n ouch! Error: stream.push() after EOF\r\n ouch! Error: stream.push() after EOF\r\n . . .\r\n */\r\n\r\n\r\n```\r\n\r\n\r\nAlso ran into an issue when trying the following in the code above:\r\n\r\n```JavaScript\r\n\r\n    output.on('error', function () {\r\n\r\n        // downstream error.  let pg know we're done with the cursor before returning the client.\r\n        query.once('close', cb);\r\n        query.close();\r\n    });\r\n\r\n```\r\n\r\nThe close event never fired because I did not read to the end of the stream.\r\n\r\nThought about adding code to emit the 'close' event as soon as the cursor closed, but figured that would be too much of a breaking change.  Instead, added a callback to the stream's close function so that developers can do something like this to avoid returning the client to the pool before the cursor is actually closed:\r\n\r\n```JavaScript\r\n\r\n    output.on('error', function () {\r\n\r\n        // downstream error.  let pg know we're done with the cursor before returning the client.\r\n        query.close(cb);\r\n    });\r\n\r\n```\r\n"},"comment":{"url":"https://api.github.com/repos/brianc/node-pg-query-stream/issues/comments/156482058","html_url":"https://github.com/brianc/node-pg-query-stream/pull/15#issuecomment-156482058","issue_url":"https://api.github.com/repos/brianc/node-pg-query-stream/issues/15","id":156482058,"user":{"login":"brianc","id":50081,"avatar_url":"https://avatars.githubusercontent.com/u/50081?v=3","gravatar_id":"","url":"https://api.github.com/users/brianc","html_url":"https://github.com/brianc","followers_url":"https://api.github.com/users/brianc/followers","following_url":"https://api.github.com/users/brianc/following{/other_user}","gists_url":"https://api.github.com/users/brianc/gists{/gist_id}","starred_url":"https://api.github.com/users/brianc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/brianc/subscriptions","organizations_url":"https://api.github.com/users/brianc/orgs","repos_url":"https://api.github.com/users/brianc/repos","events_url":"https://api.github.com/users/brianc/events{/privacy}","received_events_url":"https://api.github.com/users/brianc/received_events","type":"User","site_admin":false},"created_at":"2015-11-13T16:37:27Z","updated_at":"2015-11-13T16:37:27Z","body":"This is fantastic!  Sorry for letting this languish a bit.  I'll have this merged in today & push a new version.  Also, I fixed the failing test case in travis:\r\n\r\nhttps://github.com/brianc/node-pg-query-stream/pull/16\r\n\r\nBecause I'm bumping the version of readable stream from `1.x` to `2.x` and this module extends that class I'm gonna bump this module to `1.0` after I get these things merged in."}}