{"action":"created","issue":{"url":"https://api.github.com/repos/CartoDB/cartodb/issues/1683","labels_url":"https://api.github.com/repos/CartoDB/cartodb/issues/1683/labels{/name}","comments_url":"https://api.github.com/repos/CartoDB/cartodb/issues/1683/comments","events_url":"https://api.github.com/repos/CartoDB/cartodb/issues/1683/events","html_url":"https://github.com/CartoDB/cartodb/issues/1683","id":53742402,"number":1683,"title":"Imports user or role with big timeout","user":{"login":"Kartones","id":2085449,"avatar_url":"https://avatars.githubusercontent.com/u/2085449?v=3","gravatar_id":"","url":"https://api.github.com/users/Kartones","html_url":"https://github.com/Kartones","followers_url":"https://api.github.com/users/Kartones/followers","following_url":"https://api.github.com/users/Kartones/following{/other_user}","gists_url":"https://api.github.com/users/Kartones/gists{/gist_id}","starred_url":"https://api.github.com/users/Kartones/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Kartones/subscriptions","organizations_url":"https://api.github.com/users/Kartones/orgs","repos_url":"https://api.github.com/users/Kartones/repos","events_url":"https://api.github.com/users/Kartones/events{/privacy}","received_events_url":"https://api.github.com/users/Kartones/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/CartoDB/cartodb/labels/0+-+Backlog","name":"0 - Backlog","color":"CCCCCC"},{"url":"https://api.github.com/repos/CartoDB/cartodb/labels/backend","name":"backend","color":"d4c5f9"},{"url":"https://api.github.com/repos/CartoDB/cartodb/labels/importer","name":"importer","color":"eb6420"},{"url":"https://api.github.com/repos/CartoDB/cartodb/labels/importer-performance","name":"importer-performance","color":"006b75"}],"state":"open","locked":false,"assignee":{"login":"Kartones","id":2085449,"avatar_url":"https://avatars.githubusercontent.com/u/2085449?v=3","gravatar_id":"","url":"https://api.github.com/users/Kartones","html_url":"https://github.com/Kartones","followers_url":"https://api.github.com/users/Kartones/followers","following_url":"https://api.github.com/users/Kartones/following{/other_user}","gists_url":"https://api.github.com/users/Kartones/gists{/gist_id}","starred_url":"https://api.github.com/users/Kartones/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Kartones/subscriptions","organizations_url":"https://api.github.com/users/Kartones/orgs","repos_url":"https://api.github.com/users/Kartones/repos","events_url":"https://api.github.com/users/Kartones/events{/privacy}","received_events_url":"https://api.github.com/users/Kartones/received_events","type":"User","site_admin":false},"milestone":{"url":"https://api.github.com/repos/CartoDB/cartodb/milestones/30","labels_url":"https://api.github.com/repos/CartoDB/cartodb/milestones/30/labels","id":913526,"number":30,"title":"Matalascañas","description":"- [ ] IP Guessing on Import (@rtorre)\r\n- [ ] Url rewrite (@kartones // @javisantana)\r\n- [ ] Odissey integration in the editor. (@javisantana)\r\n- [ ] Torque Heatmap (@fdans)\r\n- [ ] Temp. Maps API specification (@rochoa)\r\n- [ ] New dashboard (@jmedina)\r\n- [ ] Limits per user / plan (@lbosque)\r\n- [ ] Apps pages (@piensaenpixel)\r\n- [ ] Editor 3.0 (@carlostallon)","creator":{"login":"saleiva","id":132149,"avatar_url":"https://avatars.githubusercontent.com/u/132149?v=3","gravatar_id":"","url":"https://api.github.com/users/saleiva","html_url":"https://github.com/saleiva","followers_url":"https://api.github.com/users/saleiva/followers","following_url":"https://api.github.com/users/saleiva/following{/other_user}","gists_url":"https://api.github.com/users/saleiva/gists{/gist_id}","starred_url":"https://api.github.com/users/saleiva/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/saleiva/subscriptions","organizations_url":"https://api.github.com/users/saleiva/orgs","repos_url":"https://api.github.com/users/saleiva/repos","events_url":"https://api.github.com/users/saleiva/events{/privacy}","received_events_url":"https://api.github.com/users/saleiva/received_events","type":"User","site_admin":false},"open_issues":37,"closed_issues":2,"state":"open","created_at":"2014-12-23T17:58:14Z","updated_at":"2015-01-08T11:48:56Z","due_on":"2015-01-21T23:00:00Z","closed_at":null},"comments":2,"created_at":"2015-01-08T11:48:56Z","updated_at":"2015-01-08T12:51:52Z","closed_at":null,"body":"We need to discuss approaches and implement a solution to avoid the increasing issues we're suffering regarding statement timeouts. Current approach of relying on the user's timeout (and increasing it manually when it is not enough) it is not scalable, not proactive (reactive and a bit ugly, first time it fails we \"fix\" it) and lately is happening on multiple fronts (raster imports, big tables, big twitter searches...).\r\n\r\n* One proposed solution in the past was having an \"importer\" role or account with huge timeout and use it for all this kind of operations.\r\nThe timeout should be in effect for both `cdb_importer` and the user's schemas, as some operations work once the table is on the destination schema (e.g. geocodings).\r\n\r\n/cc @lbosque @javisantana @juanignaciosl @zenitraM "},"comment":{"url":"https://api.github.com/repos/CartoDB/cartodb/issues/comments/69175099","html_url":"https://github.com/CartoDB/cartodb/issues/1683#issuecomment-69175099","issue_url":"https://api.github.com/repos/CartoDB/cartodb/issues/1683","id":69175099,"user":{"login":"lbosque","id":93268,"avatar_url":"https://avatars.githubusercontent.com/u/93268?v=3","gravatar_id":"","url":"https://api.github.com/users/lbosque","html_url":"https://github.com/lbosque","followers_url":"https://api.github.com/users/lbosque/followers","following_url":"https://api.github.com/users/lbosque/following{/other_user}","gists_url":"https://api.github.com/users/lbosque/gists{/gist_id}","starred_url":"https://api.github.com/users/lbosque/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lbosque/subscriptions","organizations_url":"https://api.github.com/users/lbosque/orgs","repos_url":"https://api.github.com/users/lbosque/repos","events_url":"https://api.github.com/users/lbosque/events{/privacy}","received_events_url":"https://api.github.com/users/lbosque/received_events","type":"User","site_admin":false},"created_at":"2015-01-08T12:51:52Z","updated_at":"2015-01-08T12:51:52Z","body":"The statement_timeout problems don't come only with imports. They come too with common operations we provide from the editor (altering tables, for example). The time it takes for some of these processing operations depends totally on the size or type of data which is something we cannot just control.\r\n\r\nI agree with @Kartones about using a different user for these type of cartodb processing operations (not queries that the user run). We have discussed that in the past. Having, for example, a user that inherits the db user but with a really high statement timeout could help."}}