{"action":"created","issue":{"url":"https://api.github.com/repos/brianc/node-pg-copy-streams/issues/21","labels_url":"https://api.github.com/repos/brianc/node-pg-copy-streams/issues/21/labels{/name}","comments_url":"https://api.github.com/repos/brianc/node-pg-copy-streams/issues/21/comments","events_url":"https://api.github.com/repos/brianc/node-pg-copy-streams/issues/21/events","html_url":"https://github.com/brianc/node-pg-copy-streams/issues/21","id":54945810,"number":21,"title":"Error Callback Not Initiating on PG Constraint Error","user":{"login":"BrandonMcNary","id":7433906,"avatar_url":"https://avatars.githubusercontent.com/u/7433906?v=3","gravatar_id":"","url":"https://api.github.com/users/BrandonMcNary","html_url":"https://github.com/BrandonMcNary","followers_url":"https://api.github.com/users/BrandonMcNary/followers","following_url":"https://api.github.com/users/BrandonMcNary/following{/other_user}","gists_url":"https://api.github.com/users/BrandonMcNary/gists{/gist_id}","starred_url":"https://api.github.com/users/BrandonMcNary/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/BrandonMcNary/subscriptions","organizations_url":"https://api.github.com/users/BrandonMcNary/orgs","repos_url":"https://api.github.com/users/BrandonMcNary/repos","events_url":"https://api.github.com/users/BrandonMcNary/events{/privacy}","received_events_url":"https://api.github.com/users/BrandonMcNary/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2015-01-20T21:55:01Z","updated_at":"2015-01-27T06:04:19Z","closed_at":null,"body":"Hi Brian,\r\n\r\nThank you so much for this tool. It easily streams 40,000 rows + into our Postgres database. I have one issue though, I'm unable to get it to throw errors when there is a problem. Certain tables within our database have unique constraints. When a user attempts to upload a duplicated file, the database throws an error, but I can't get this to bubble back to the user. I've tinkered with the error callback provided, but for whatever reason it doesn't trigger when a database error is thrown. \r\n\r\nDo you have any recommendations? Please let me know if I'm doing something wrong here. \r\n\r\nThanks again!\r\nBrandon\r\n\r\nI'm using this code:\r\n\r\npg.connect(connectionString, function(err, client, done) {\r\n        if(err) {\r\n            return console.error('could not connect to postgres', err);\r\n        }\r\n        var stream = client.query(copyFrom(query));\r\n        var s = new Readable;\r\n        s.push(csv);\r\n        s.push(null);\r\n        s.pipe(stream).on('finish', function () {\r\n            res.send(200);\r\n            done();\r\n        }).on('error', function () {\r\n            res.send(500);\r\n            done();\r\n        });\r\n    });\r\n};\r\n"},"comment":{"url":"https://api.github.com/repos/brianc/node-pg-copy-streams/issues/comments/71594273","html_url":"https://github.com/brianc/node-pg-copy-streams/issues/21#issuecomment-71594273","issue_url":"https://api.github.com/repos/brianc/node-pg-copy-streams/issues/21","id":71594273,"user":{"login":"BrandonMcNary","id":7433906,"avatar_url":"https://avatars.githubusercontent.com/u/7433906?v=3","gravatar_id":"","url":"https://api.github.com/users/BrandonMcNary","html_url":"https://github.com/BrandonMcNary","followers_url":"https://api.github.com/users/BrandonMcNary/followers","following_url":"https://api.github.com/users/BrandonMcNary/following{/other_user}","gists_url":"https://api.github.com/users/BrandonMcNary/gists{/gist_id}","starred_url":"https://api.github.com/users/BrandonMcNary/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/BrandonMcNary/subscriptions","organizations_url":"https://api.github.com/users/BrandonMcNary/orgs","repos_url":"https://api.github.com/users/BrandonMcNary/repos","events_url":"https://api.github.com/users/BrandonMcNary/events{/privacy}","received_events_url":"https://api.github.com/users/BrandonMcNary/received_events","type":"User","site_admin":false},"created_at":"2015-01-27T06:04:19Z","updated_at":"2015-01-27T06:04:19Z","body":"Hi Brian,\r\n\r\nThanks for the quick reply. I was trying to figure out how to achieve this, as a front to back sample would be quite a bit of work. Is there something specifically you'd like to see? The code I pasted was a portion of the uploadData function. I pasted the rest of the server code surrounding the uploadData function below. I've also included the angular service which handles the HTTP request to the server after a user has submitted the file. Sorry it's not the sample you wanted. If you have any suggestions to get you what you're looking for, please let me know.\r\n\r\nKind Regards,\r\nBrandon\r\n\r\n'use strict'\r\nvar knex = require(__dirname+'/../../knexconfig.js').knex;\r\nvar connectionString = require(__dirname+'/../../knexconfig.js').connectionString;\r\nvar pg = require('pg');\r\nvar copyFrom = require('pg-copy-streams').from;\r\nvar fs = require ('fs');\r\nvar Readable = require('stream').Readable;\r\n\r\nvar uploadData = function(req, res){\r\n    var csv = ConvertToCSV(req.body),\r\n        table,\r\n        trendType = (req.param('trendType') === 'Energy') ? 'electric' : 'power';\r\n\r\n    if (req.param('uploadType') === 'dataInterval'){\r\n        table = trendType + 'datainterval';\r\n    }\r\n    if (req.param('uploadType') === 'waterInterval'){\r\n        table = 'waterdata';\r\n    }\r\n    var query = 'COPY ' + table + ' FROM STDIN';\r\n    pg.connect(connectionString, function(err, client, done) {\r\n        if(err) {\r\n            return console.error('could not connect to postgres', err);\r\n        }\r\n        var stream = client.query(copyFrom(query));\r\n        var s = new Readable;\r\n        s.push(csv);\r\n        s.push(null);\r\n\r\n        s.pipe(stream).on('finish', function () {\r\n            res.send(200);\r\n            done();\r\n        }).on('error', function () {\r\n            res.send(500);\r\n            done();\r\n        });\r\n    });\r\n};\r\n\r\nvar ConvertToCSV = function(objArray) {\r\n    var array = typeof objArray != 'object' ? JSON.parse(objArray) : objArray;\r\n    var str = '';\r\n    for (var i = 0; i < array.length; i++) {\r\n        var line = '';\r\n        for (var index in array[i]) {\r\n            if (line != '') line += '\\t'\r\n            line += array[i][index];\r\n        }\r\n        str += line + '\\r\\n';\r\n    }\r\n    return str;\r\n};\r\n\r\n--- End Server\r\n-- Start Angular HTTP Service\r\n//HTTP query launched when a user submits a CSV file (fileObject is the CSV)\r\n\r\n return helpers = {\r\n    saveEnergyUse: function(fileObject, trendType, uploadType, farmId, companyId){\r\n      var deferred = $q.defer();\r\n      $http({\r\n        method: 'POST',\r\n        url: '/api/uploadData/' + uploadType + '/' + trendType,\r\n        data: fileObject,\r\n      }).success(function(record){\r\n        deferred.resolve(record);\r\n      }).error(function(info, status){\r\n        deferred.reject('Error', info, 'Status', status);\r\n      console.error(info);\r\n      });\r\n      return deferred.promise;\r\n    }\r\n}\r\n\r\n--- END Angular Service\r\n"}}