{"action":"created","issue":{"url":"https://api.github.com/repos/ciconia/lydown/issues/5","labels_url":"https://api.github.com/repos/ciconia/lydown/issues/5/labels{/name}","comments_url":"https://api.github.com/repos/ciconia/lydown/issues/5/comments","events_url":"https://api.github.com/repos/ciconia/lydown/issues/5/events","html_url":"https://github.com/ciconia/lydown/issues/5","id":97266049,"number":5,"title":"Parallelize source file processing","user":{"login":"ciconia","id":291,"avatar_url":"https://avatars.githubusercontent.com/u/291?v=3","gravatar_id":"","url":"https://api.github.com/users/ciconia","html_url":"https://github.com/ciconia","followers_url":"https://api.github.com/users/ciconia/followers","following_url":"https://api.github.com/users/ciconia/following{/other_user}","gists_url":"https://api.github.com/users/ciconia/gists{/gist_id}","starred_url":"https://api.github.com/users/ciconia/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ciconia/subscriptions","organizations_url":"https://api.github.com/users/ciconia/orgs","repos_url":"https://api.github.com/users/ciconia/repos","events_url":"https://api.github.com/users/ciconia/events{/privacy}","received_events_url":"https://api.github.com/users/ciconia/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/ciconia/lydown/labels/enhancement","name":"enhancement","color":"84b6eb"}],"state":"open","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/ciconia/lydown/milestones/2","html_url":"https://github.com/ciconia/lydown/milestones/0.10%20Itay","labels_url":"https://api.github.com/repos/ciconia/lydown/milestones/2/labels","id":1225893,"number":2,"title":"0.10 Itay","description":"Tous qu'il faudrait pour produire les partitions et parties.","creator":{"login":"ciconia","id":291,"avatar_url":"https://avatars.githubusercontent.com/u/291?v=3","gravatar_id":"","url":"https://api.github.com/users/ciconia","html_url":"https://github.com/ciconia","followers_url":"https://api.github.com/users/ciconia/followers","following_url":"https://api.github.com/users/ciconia/following{/other_user}","gists_url":"https://api.github.com/users/ciconia/gists{/gist_id}","starred_url":"https://api.github.com/users/ciconia/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ciconia/subscriptions","organizations_url":"https://api.github.com/users/ciconia/orgs","repos_url":"https://api.github.com/users/ciconia/repos","events_url":"https://api.github.com/users/ciconia/events{/privacy}","received_events_url":"https://api.github.com/users/ciconia/received_events","type":"User","site_admin":false},"open_issues":7,"closed_issues":2,"state":"open","created_at":"2015-07-25T23:03:27Z","updated_at":"2015-09-08T05:09:08Z","due_on":"2015-08-25T22:00:00Z","closed_at":null},"comments":4,"created_at":"2015-07-25T23:22:08Z","updated_at":"2015-09-12T07:30:11Z","closed_at":null,"body":"When processing a directory, things can be parallelized:\r\n\r\n* Parsing individual files can be performed by a thread pool. Generating an AST and event stream for each file can be completely parallelized. The actual generation of lilypond code can be serialized using a critical section or some other lock.\r\n\r\n* This actually should be relatively simple:\r\n  * Work#process_directory starts a worker thread pool and adds each encountered file to the worker queue.\r\n  * Workers pull file names from the worker queue and process them using process_lydown_file.\r\n  * Work#process_lydown_file uses a critical section or other lock to synchronize the last stage of processing, the actual translation to lilypond code is serialized."},"comment":{"url":"https://api.github.com/repos/ciconia/lydown/issues/comments/139738397","html_url":"https://github.com/ciconia/lydown/issues/5#issuecomment-139738397","issue_url":"https://api.github.com/repos/ciconia/lydown/issues/5","id":139738397,"user":{"login":"ciconia","id":291,"avatar_url":"https://avatars.githubusercontent.com/u/291?v=3","gravatar_id":"","url":"https://api.github.com/users/ciconia","html_url":"https://github.com/ciconia","followers_url":"https://api.github.com/users/ciconia/followers","following_url":"https://api.github.com/users/ciconia/following{/other_user}","gists_url":"https://api.github.com/users/ciconia/gists{/gist_id}","starred_url":"https://api.github.com/users/ciconia/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ciconia/subscriptions","organizations_url":"https://api.github.com/users/ciconia/orgs","repos_url":"https://api.github.com/users/ciconia/repos","events_url":"https://api.github.com/users/ciconia/events{/privacy}","received_events_url":"https://api.github.com/users/ciconia/received_events","type":"User","site_admin":false},"created_at":"2015-09-12T07:30:11Z","updated_at":"2015-09-12T07:30:11Z","body":"Another possibility of parallelizing compilation is to take advantage of movement page breaks, which when rendered create separate \\bookPart sections. Each section is rendered separately, then compiled separately. The result is then combined into a single PDF using [pdftk](https://github.com/webkrak/pdftk)."}}