{"action":"created","issue":{"url":"https://api.github.com/repos/chapmanb/bcbio-nextgen/issues/943","labels_url":"https://api.github.com/repos/chapmanb/bcbio-nextgen/issues/943/labels{/name}","comments_url":"https://api.github.com/repos/chapmanb/bcbio-nextgen/issues/943/comments","events_url":"https://api.github.com/repos/chapmanb/bcbio-nextgen/issues/943/events","html_url":"https://github.com/chapmanb/bcbio-nextgen/issues/943","id":96404108,"number":943,"title":"Cutadapt only using a single node","user":{"login":"mdeboth","id":6434159,"avatar_url":"https://avatars.githubusercontent.com/u/6434159?v=3","gravatar_id":"","url":"https://api.github.com/users/mdeboth","html_url":"https://github.com/mdeboth","followers_url":"https://api.github.com/users/mdeboth/followers","following_url":"https://api.github.com/users/mdeboth/following{/other_user}","gists_url":"https://api.github.com/users/mdeboth/gists{/gist_id}","starred_url":"https://api.github.com/users/mdeboth/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mdeboth/subscriptions","organizations_url":"https://api.github.com/users/mdeboth/orgs","repos_url":"https://api.github.com/users/mdeboth/repos","events_url":"https://api.github.com/users/mdeboth/events{/privacy}","received_events_url":"https://api.github.com/users/mdeboth/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":12,"created_at":"2015-07-21T19:58:56Z","updated_at":"2015-07-24T09:51:33Z","closed_at":"2015-07-22T14:18:56Z","body":"I recently did a clean re-install of bcbio and can't seem to get Cutadapt to parallelize correctly... it only uses a single compute node. This isn't a re-run of a previous project; there aren't any checkpoint files. Launch bcbio with 128 cores...\r\n`bcbio_nextgen.py /scratch/mdeboth/rock/rock/config/rock.yaml -t ipython -s torque -q batch -n 128 --retries 1 --timeout 2880 --tag rock`\r\n\r\nTrimming starts with 128 jobs, but they're assigned to a single node & take a day or two to complete. Once it passes trimming, alignment gets distributed across nodes, so I don't think it's a network issue. Any ideas? Thanks in advance!\r\n```\r\n[2015-07-20T21:54Z] pnap-pe2-s15: Resource requests: cutadapt, picard; memory: 3.50; cores: 1, 1\r\n[2015-07-20T21:54Z] pnap-pe2-s15: Configuring 128 jobs to run, using 1 cores each with 3.50g of memory reserved for each job\r\n[ProfileCreate] Generating default config file: u'/scratch/mdeboth/rock/rock/work/log/ipython/ipython_config.py'\r\n[ProfileCreate] Generating default config file: u'/scratch/mdeboth/rock/rock/work/log/ipython/ipython_kernel_config.py'\r\n[ProfileCreate] Generating default config file: u'/scratch/mdeboth/rock/rock/work/log/ipython/ipython_console_config.py'\r\n[ProfileCreate] Generating default config file: u'/scratch/mdeboth/rock/rock/work/log/ipython/ipcontroller_config.py'\r\n[ProfileCreate] Generating default config file: u'/scratch/mdeboth/rock/rock/work/log/ipython/ipengine_config.py'\r\n[ProfileCreate] Generating default config file: u'/scratch/mdeboth/rock/rock/work/log/ipython/ipcluster_config.py'\r\n[ProfileCreate] Generating default config file: u'/scratch/mdeboth/rock/rock/work/log/ipython/iplogger_config.py'\r\n2015-07-20 14:54:32.854 [IPClusterStart] Config changed:\r\n2015-07-20 14:54:32.854 [IPClusterStart] {'BcbioTORQUEEngineSetLauncher': {'mem': '3.50', 'cores': 1, 'tag': 'rock', 'numengines': 15, 'resources': 'walltime=48:00:00'}, 'IPClusterEngines': {'early_shutdown': 240}, 'Application': {'log_level': 10}, 'ProfileDir': {'locatio\r\nn': u'/scratch/mdeboth/rock/rock/work/log/ipython'}, 'BaseParallelApplication': {'log_to_file': True, 'cluster_id': u'101a376e-9079-4fb5-9ba2-0659705c627d'}, 'TORQUELauncher': {'queue': 'batch'}, 'BcbioTORQUEControllerLauncher': {'mem': '', 'cores': 1, 'tag': 'rock', 'res\r\nources': 'walltime=48:00:00'}, 'IPClusterStart': {'delay': 10, 'n': 9, 'daemonize': True, 'engine_launcher_class': u'cluster_helper.cluster.BcbioTORQUEEngineSetLauncher', 'controller_launcher_class': u'cluster_helper.cluster.BcbioTORQUEControllerLauncher'}}\r\n2015-07-20 14:54:32.861 [IPClusterStart] Using existing profile dir: u'/scratch/mdeboth/rock/rock/work/log/ipython'\r\n2015-07-20 14:54:32.862 [IPClusterStart] Searching path [u'/scratch/mdeboth/rock/rock/work', u'/scratch/mdeboth/rock/rock/work/log/ipython', '/usr/local/etc/ipython', '/etc/ipython'] for config files\r\n2015-07-20 14:54:32.862 [IPClusterStart] Attempting to load config file: ipython_config.py\r\n2015-07-20 14:54:32.863 [IPClusterStart] Loaded config file: /scratch/mdeboth/rock/rock/work/log/ipython/ipython_config.py\r\n2015-07-20 14:54:32.863 [IPClusterStart] Attempting to load config file: ipcluster_101a376e_9079_4fb5_9ba2_0659705c627d_config.py\r\n2015-07-20 14:54:32.864 [IPClusterStart] Loaded config file: /scratch/mdeboth/rock/rock/work/log/ipython/ipcontroller_config.py\r\n2015-07-20 14:54:32.864 [IPClusterStart] Attempting to load config file: ipcluster_101a376e_9079_4fb5_9ba2_0659705c627d_config.py\r\n2015-07-20 14:54:32.865 [IPClusterStart] Loaded config file: /scratch/mdeboth/rock/rock/work/log/ipython/ipengine_config.py\r\n2015-07-20 14:54:32.865 [IPClusterStart] Attempting to load config file: ipcluster_101a376e_9079_4fb5_9ba2_0659705c627d_config.py\r\n2015-07-20 14:54:32.866 [IPClusterStart] Loaded config file: /scratch/mdeboth/rock/rock/work/log/ipython/ipcluster_config.py\r\n[2015-07-20T21:57Z] pnap-pe2-s15: Timing: organize samples\r\n[2015-07-20T21:57Z] pnap-pe2-s15: ipython: organize_samples\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Using input YAML configuration: /scratch/mdeboth/rock/rock/config/rock.yaml\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Checking sample YAML configuration: /scratch/mdeboth/rock/rock/config/rock.yaml\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Testing minimum versions of installed programs\r\n[2015-07-20T21:57Z] pnap-pe2-s15: Timing: adapter trimming\r\n[2015-07-20T21:57Z] pnap-pe2-s15: ipython: prepare_sample\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s2_water_r2\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s2_fas_r1\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s4_t299_r3\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s6_dmso_r4\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s3_fas_r4\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s2_t343_r3\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s3_y27_r4\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s6_fas_r1\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s4_y27_r4\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s6_dmso_r1\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s2_fas_r2\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s4_fas_r2\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s2_t343_r4\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s2_t343_r2\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s1_fas_r2\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s5_y27_r3\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s6_t343_r4\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s5_fas_r4\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s4_y27_r3\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s3_t343_r3\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s1_t343_r3\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s3_t343_r1\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s2_y27_r3\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s1_fas_r4\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s1_t343_r1\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s5_fas_r3\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s5_t299_r4\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s3_fas_r2\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s2_water_r1\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s6_water_r4\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s4_water_r2\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s5_dmso_r3\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s3_y27_r3\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s2_y27_r2\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s1_t299_r4\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s5_dmso_r1\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s1_fas_r1\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s4_t299_r4\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s2_t343_r1\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s3_y27_r2\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s5_y27_r4\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s1_dmso_r1\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s4_t299_r2\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s5_dmso_r4\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s5_t299_r3\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s1_water_r2\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s4_water_r3\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s1_t343_r4\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s5_t299_r1\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s6_y27_r3\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s2_fas_r4\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s2_water_r3\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s2_t299_r2\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s5_y27_r1\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s6_y27_r1\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s3_t299_r3\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s3_t343_r2\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s3_t299_r4\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s3_water_r3\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s1_fas_r3\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s4_t299_r1\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s3_dmso_r3\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s4_t343_r3\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s3_water_r2\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s4_dmso_r4\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s6_fas_r4\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s5_water_r4\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s4_dmso_r1\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s1_dmso_r3\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s3_water_r4\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s1_t299_r2\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s6_dmso_r3\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s6_t299_r1\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s5_water_r2\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s5_t299_r2\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s1_y27_r1\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s4_t343_r1\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s1_water_r1\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s1_dmso_r2\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s6_dmso_r2\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s6_fas_r2\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s3_y27_r1\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s2_dmso_r1\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s5_dmso_r2\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s2_water_r4\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s3_dmso_r1\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s1_water_r3\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s1_t299_r1\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s4_t343_r4\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s3_fas_r3\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s1_t299_r3\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s1_y27_r2\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s1_water_r4\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s2_dmso_r4\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s2_t299_r4\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s5_y27_r2\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s6_t343_r2\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s3_dmso_r4\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s4_t343_r2\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s2_fas_r3\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s6_water_r2\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s3_t343_r4\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s6_t299_r3\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s3_dmso_r2\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s6_water_r3\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s4_dmso_r3\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s5_t343_r2\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s4_fas_r1\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s4_dmso_r2\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s6_y27_r4\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s5_t343_r4\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s2_y27_r1\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s3_fas_r1\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s6_t343_r3\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s5_fas_r1\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s3_t299_r1\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s4_fas_r3\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s2_t299_r1\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s1_y27_r4\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s4_fas_r4\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s5_water_r3\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s4_water_r1\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s5_water_r1\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s6_water_r1\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s2_t299_r3\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s1_y27_r3\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s2_dmso_r2\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s1_dmso_r4\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s6_y27_r2\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s3_t299_r2\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s5_fas_r2\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s6_t343_r1\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s5_t343_r3\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s2_dmso_r3\r\n[2015-07-20T21:57Z] pnap-pe5-s15: Preparing s6_fas_r3\r\n[2015-07-20T21:57Z] pnap-pe6-s08: Preparing s4_y27_r2\r\n```"},"comment":{"url":"https://api.github.com/repos/chapmanb/bcbio-nextgen/issues/comments/124459230","html_url":"https://github.com/chapmanb/bcbio-nextgen/issues/943#issuecomment-124459230","issue_url":"https://api.github.com/repos/chapmanb/bcbio-nextgen/issues/943","id":124459230,"user":{"login":"chapmanb","id":39391,"avatar_url":"https://avatars.githubusercontent.com/u/39391?v=3","gravatar_id":"","url":"https://api.github.com/users/chapmanb","html_url":"https://github.com/chapmanb","followers_url":"https://api.github.com/users/chapmanb/followers","following_url":"https://api.github.com/users/chapmanb/following{/other_user}","gists_url":"https://api.github.com/users/chapmanb/gists{/gist_id}","starred_url":"https://api.github.com/users/chapmanb/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/chapmanb/subscriptions","organizations_url":"https://api.github.com/users/chapmanb/orgs","repos_url":"https://api.github.com/users/chapmanb/repos","events_url":"https://api.github.com/users/chapmanb/events{/privacy}","received_events_url":"https://api.github.com/users/chapmanb/received_events","type":"User","site_admin":false},"created_at":"2015-07-24T09:51:33Z","updated_at":"2015-07-24T09:51:33Z","body":"Matt -- good catch, thanks for the fix. I pushed this to bcbio so hopefully it'll work for you in future updates as well. Much appreciated."}}