{"action":"created","issue":{"url":"https://api.github.com/repos/18F/midas/issues/813","labels_url":"https://api.github.com/repos/18F/midas/issues/813/labels{/name}","comments_url":"https://api.github.com/repos/18F/midas/issues/813/comments","events_url":"https://api.github.com/repos/18F/midas/issues/813/events","html_url":"https://github.com/18F/midas/issues/813","id":81581226,"number":813,"title":"Performance on /tasks is slow","user":{"login":"dhcole","id":170641,"avatar_url":"https://avatars.githubusercontent.com/u/170641?v=3","gravatar_id":"","url":"https://api.github.com/users/dhcole","html_url":"https://github.com/dhcole","followers_url":"https://api.github.com/users/dhcole/followers","following_url":"https://api.github.com/users/dhcole/following{/other_user}","gists_url":"https://api.github.com/users/dhcole/gists{/gist_id}","starred_url":"https://api.github.com/users/dhcole/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dhcole/subscriptions","organizations_url":"https://api.github.com/users/dhcole/orgs","repos_url":"https://api.github.com/users/dhcole/repos","events_url":"https://api.github.com/users/dhcole/events{/privacy}","received_events_url":"https://api.github.com/users/dhcole/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/18F/midas/labels/in%20progress","name":"in progress","color":"ededed"},{"url":"https://api.github.com/repos/18F/midas/labels/Open%20Opps","name":"Open Opps","color":"4488aa"}],"state":"open","locked":false,"assignee":{"login":"dhcole","id":170641,"avatar_url":"https://avatars.githubusercontent.com/u/170641?v=3","gravatar_id":"","url":"https://api.github.com/users/dhcole","html_url":"https://github.com/dhcole","followers_url":"https://api.github.com/users/dhcole/followers","following_url":"https://api.github.com/users/dhcole/following{/other_user}","gists_url":"https://api.github.com/users/dhcole/gists{/gist_id}","starred_url":"https://api.github.com/users/dhcole/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dhcole/subscriptions","organizations_url":"https://api.github.com/users/dhcole/orgs","repos_url":"https://api.github.com/users/dhcole/repos","events_url":"https://api.github.com/users/dhcole/events{/privacy}","received_events_url":"https://api.github.com/users/dhcole/received_events","type":"User","site_admin":false},"milestone":null,"comments":9,"created_at":"2015-05-27T19:05:57Z","updated_at":"2015-06-03T20:56:23Z","closed_at":null,"body":"Just a general capture of thought about performance on the tasks page.\r\n\r\nPage loads on https://midas.18f.us/tasks are taking about 4-5 seconds. This is down by about 50% due to the #656 Tag Associations refactor, but still too high.\r\n\r\n![screen shot 2015-05-27 at 2 53 56 pm](https://cloud.githubusercontent.com/assets/170641/7844723/5e4a13e2-0480-11e5-9ec2-47423627ad96.png)\r\n\r\nThe delay on this page is mostly due to the `/api/tasks` endpoint. The manual association between tagEntities and other models was inefficient. \r\n\r\nHere are additional sources of slowness:\r\n\r\n- `/api/file/get/*` used to require querying file blobs from the database. In dev, this is moved to disk or S3, but it's not rolled out to production yet. This accounts for 17% of request time: \r\n![screen shot 2015-05-27 at 2 55 52 pm](https://cloud.githubusercontent.com/assets/170641/7844777/eb8b89d4-0480-11e5-9325-577dd74e86ea.png)\r\n- Sessions are stored in the main postgres DB (That's what all the `head` requests in the previous chart show). We can move sessions to a separate database to help with this. Sails / Express recommend redis, which should be available in AWS through the Elasticache service\r\n- The `/api/task` endpoint is currently returning more data then necessary. We could limit these requests with pagination. There is an infinite scroll implementation of pagination available in the front-end code, however, we need to confirm that this works well with the search and filters on the `/tasks` page\r\n- There is a `like` query on the `/api/task` endpoint that seems to cause the bulk of slowness now. We could track down where this query is coming from and figure out how to optimize it"},"comment":{"url":"https://api.github.com/repos/18F/midas/issues/comments/108614071","html_url":"https://github.com/18F/midas/issues/813#issuecomment-108614071","issue_url":"https://api.github.com/repos/18F/midas/issues/813","id":108614071,"user":{"login":"dhcole","id":170641,"avatar_url":"https://avatars.githubusercontent.com/u/170641?v=3","gravatar_id":"","url":"https://api.github.com/users/dhcole","html_url":"https://github.com/dhcole","followers_url":"https://api.github.com/users/dhcole/followers","following_url":"https://api.github.com/users/dhcole/following{/other_user}","gists_url":"https://api.github.com/users/dhcole/gists{/gist_id}","starred_url":"https://api.github.com/users/dhcole/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dhcole/subscriptions","organizations_url":"https://api.github.com/users/dhcole/orgs","repos_url":"https://api.github.com/users/dhcole/repos","events_url":"https://api.github.com/users/dhcole/events{/privacy}","received_events_url":"https://api.github.com/users/dhcole/received_events","type":"User","site_admin":false},"created_at":"2015-06-03T20:56:23Z","updated_at":"2015-06-03T20:56:23Z","body":"No real SSD performance improvement... onto the web server:\r\n\r\n```\r\ndavidcole@dc:~/dev/midas$ time curl https://openopps-staging.18f.us/api/task > /dev/null\r\n  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current\r\n                                 Dload  Upload   Total   Spent    Left  Speed\r\n100  255k  100  255k    0     0  54122      0  0:00:04  0:00:04 --:--:-- 72260\r\n\r\nreal\t0m4.834s\r\nuser\t0m0.033s\r\nsys\t0m0.015s\r\n```"}}