{"action":"opened","issue":{"url":"https://api.github.com/repos/datamade/parserator/issues/18","labels_url":"https://api.github.com/repos/datamade/parserator/issues/18/labels{/name}","comments_url":"https://api.github.com/repos/datamade/parserator/issues/18/comments","events_url":"https://api.github.com/repos/datamade/parserator/issues/18/events","html_url":"https://github.com/datamade/parserator/issues/18","id":67615625,"number":18,"title":"Native support for parallelization","user":{"login":"AbeHandler","id":1252925,"avatar_url":"https://avatars.githubusercontent.com/u/1252925?v=3","gravatar_id":"","url":"https://api.github.com/users/AbeHandler","html_url":"https://github.com/AbeHandler","followers_url":"https://api.github.com/users/AbeHandler/followers","following_url":"https://api.github.com/users/AbeHandler/following{/other_user}","gists_url":"https://api.github.com/users/AbeHandler/gists{/gist_id}","starred_url":"https://api.github.com/users/AbeHandler/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AbeHandler/subscriptions","organizations_url":"https://api.github.com/users/AbeHandler/orgs","repos_url":"https://api.github.com/users/AbeHandler/repos","events_url":"https://api.github.com/users/AbeHandler/events{/privacy}","received_events_url":"https://api.github.com/users/AbeHandler/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2015-04-10T15:05:16Z","updated_at":"2015-04-10T15:05:16Z","closed_at":null,"body":"Parserator starts lagging when training on large documents -- and when parsing large documents. It would be nice if Parserator used some of the Python parallelization modules to take advantage of multiple cores, when available. Right now, I'm working around this (somewhat) using the parallel command. But it would be great if you could just pass parserator a -P flag and it would parallelize for you.\r\n\r\n    parallel python parse_documents.py ::: $(find path/to/corpus/*_text.txt -type f)"}}