{"action":"created","issue":{"url":"https://api.github.com/repos/Andersbakken/rtags/issues/529","labels_url":"https://api.github.com/repos/Andersbakken/rtags/issues/529/labels{/name}","comments_url":"https://api.github.com/repos/Andersbakken/rtags/issues/529/comments","events_url":"https://api.github.com/repos/Andersbakken/rtags/issues/529/events","html_url":"https://github.com/Andersbakken/rtags/issues/529","id":118066751,"number":529,"title":"rdm only uses 1 cpu even when setting job count","user":{"login":"ismacaulay","id":1445333,"avatar_url":"https://avatars.githubusercontent.com/u/1445333?v=3","gravatar_id":"","url":"https://api.github.com/users/ismacaulay","html_url":"https://github.com/ismacaulay","followers_url":"https://api.github.com/users/ismacaulay/followers","following_url":"https://api.github.com/users/ismacaulay/following{/other_user}","gists_url":"https://api.github.com/users/ismacaulay/gists{/gist_id}","starred_url":"https://api.github.com/users/ismacaulay/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ismacaulay/subscriptions","organizations_url":"https://api.github.com/users/ismacaulay/orgs","repos_url":"https://api.github.com/users/ismacaulay/repos","events_url":"https://api.github.com/users/ismacaulay/events{/privacy}","received_events_url":"https://api.github.com/users/ismacaulay/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":10,"created_at":"2015-11-20T15:43:47Z","updated_at":"2015-12-08T17:28:40Z","closed_at":null,"body":"What was done:\r\n- start rdm with `rdm -C -j 50` to clean out the project and start it with 50 jobs\r\n\r\nWhat was seen:\r\n- after executing `rc -J <path to compilation db>` rdm is using about 35% of my cpu % (according to htop) and typically only cpu1 is doing any work.\r\n\r\nWhat was expected:\r\n- I would have expected rdm to use all available cpu resources while indexing the compilation database. \r\n\r\nI am using a good computer (intel xeon e5 @ 3.50, 6 cores, 32 gb ram, ssd, Ubuntu 14.04) and the compilation database is ~15000 entries. This causes the indexing to take hours. Let me know if there is anything I do to try to help speed this up."},"comment":{"url":"https://api.github.com/repos/Andersbakken/rtags/issues/comments/162953284","html_url":"https://github.com/Andersbakken/rtags/issues/529#issuecomment-162953284","issue_url":"https://api.github.com/repos/Andersbakken/rtags/issues/529","id":162953284,"user":{"login":"Andersbakken","id":73034,"avatar_url":"https://avatars.githubusercontent.com/u/73034?v=3","gravatar_id":"","url":"https://api.github.com/users/Andersbakken","html_url":"https://github.com/Andersbakken","followers_url":"https://api.github.com/users/Andersbakken/followers","following_url":"https://api.github.com/users/Andersbakken/following{/other_user}","gists_url":"https://api.github.com/users/Andersbakken/gists{/gist_id}","starred_url":"https://api.github.com/users/Andersbakken/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Andersbakken/subscriptions","organizations_url":"https://api.github.com/users/Andersbakken/orgs","repos_url":"https://api.github.com/users/Andersbakken/repos","events_url":"https://api.github.com/users/Andersbakken/events{/privacy}","received_events_url":"https://api.github.com/users/Andersbakken/received_events","type":"User","site_admin":false},"created_at":"2015-12-08T17:28:40Z","updated_at":"2015-12-08T17:28:40Z","body":"Sorry about the delay. There's an unusual amount of time spent in visiting the AST after parsing it. E.g. 516/34109/2ms means that clang used only 516 ms to parse the file whereas rp used 34 seconds to visit the cursors.\r\n\r\nI believe we have a problem communicating back to rdm over the unix socket and that it times out whenever it tries to query back. \r\n\r\nAre rc commands working? \r\n\r\nE.g. Can you run:\r\n\r\nrc -w \r\n\r\nand get the output in a reasonable amount of time? \r\n\r\nI also added some additional timing to the output but it very much seems like the delay is a direct function of the number of files queried.\r\n\r\nIn short rp has to \"phone home\" to rdm whenever we see a file that hasn't been indexed yet. "}}