{"action":"created","issue":{"url":"https://api.github.com/repos/EdinburghGenomics/Analysis-Driver/issues/55","labels_url":"https://api.github.com/repos/EdinburghGenomics/Analysis-Driver/issues/55/labels{/name}","comments_url":"https://api.github.com/repos/EdinburghGenomics/Analysis-Driver/issues/55/comments","events_url":"https://api.github.com/repos/EdinburghGenomics/Analysis-Driver/issues/55/events","html_url":"https://github.com/EdinburghGenomics/Analysis-Driver/issues/55","id":109464848,"number":55,"title":"Executor overhaul","user":{"login":"mwhamgenomics","id":12951686,"avatar_url":"https://avatars.githubusercontent.com/u/12951686?v=3","gravatar_id":"","url":"https://api.github.com/users/mwhamgenomics","html_url":"https://github.com/mwhamgenomics","followers_url":"https://api.github.com/users/mwhamgenomics/followers","following_url":"https://api.github.com/users/mwhamgenomics/following{/other_user}","gists_url":"https://api.github.com/users/mwhamgenomics/gists{/gist_id}","starred_url":"https://api.github.com/users/mwhamgenomics/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mwhamgenomics/subscriptions","organizations_url":"https://api.github.com/users/mwhamgenomics/orgs","repos_url":"https://api.github.com/users/mwhamgenomics/repos","events_url":"https://api.github.com/users/mwhamgenomics/events{/privacy}","received_events_url":"https://api.github.com/users/mwhamgenomics/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/EdinburghGenomics/Analysis-Driver/labels/enhancement","name":"enhancement","color":"bfe5bf"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-10-02T10:06:44Z","updated_at":"2015-10-04T07:59:14Z","closed_at":null,"body":"The usage of the Executor classes is a bit complex at the moment. An ideal situation would be:\r\n- build up a list of Bash commands\r\n- call, e.g, executor.execute(bash_commands, cluster=True, block=False)\r\n- if cluster == True, then use PBSWriter/ClusterExecutor\r\n  - if len(bash_commands) > 1, then write a  job array\r\n\r\nWe should also consolidate the Threaded exception-catching from quality_control.GenotypeValidation\r\n"},"comment":{"url":"https://api.github.com/repos/EdinburghGenomics/Analysis-Driver/issues/comments/145325168","html_url":"https://github.com/EdinburghGenomics/Analysis-Driver/issues/55#issuecomment-145325168","issue_url":"https://api.github.com/repos/EdinburghGenomics/Analysis-Driver/issues/55","id":145325168,"user":{"login":"tcezard","id":650891,"avatar_url":"https://avatars.githubusercontent.com/u/650891?v=3","gravatar_id":"","url":"https://api.github.com/users/tcezard","html_url":"https://github.com/tcezard","followers_url":"https://api.github.com/users/tcezard/followers","following_url":"https://api.github.com/users/tcezard/following{/other_user}","gists_url":"https://api.github.com/users/tcezard/gists{/gist_id}","starred_url":"https://api.github.com/users/tcezard/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tcezard/subscriptions","organizations_url":"https://api.github.com/users/tcezard/orgs","repos_url":"https://api.github.com/users/tcezard/repos","events_url":"https://api.github.com/users/tcezard/events{/privacy}","received_events_url":"https://api.github.com/users/tcezard/received_events","type":"User","site_admin":false},"created_at":"2015-10-04T07:59:14Z","updated_at":"2015-10-04T07:59:14Z","body":"You should allow arbitrary parameters to be passed with kwargs in the executor.\r\nI don't you need every jobs to specify if they should be run on the cluster or locally but the executor should make that decision based on the config file.\r\nA mechanism for bypassing this might be useful ie: force_ job_execution = 'local'\r\n\r\n"}}