{"action":"created","issue":{"url":"https://api.github.com/repos/BD2KGenomics/toil/issues/598","labels_url":"https://api.github.com/repos/BD2KGenomics/toil/issues/598/labels{/name}","comments_url":"https://api.github.com/repos/BD2KGenomics/toil/issues/598/comments","events_url":"https://api.github.com/repos/BD2KGenomics/toil/issues/598/events","html_url":"https://github.com/BD2KGenomics/toil/issues/598","id":120253911,"number":598,"title":"Running into some resource allocation problems","user":{"login":"hannes-ucsc","id":5143256,"avatar_url":"https://avatars.githubusercontent.com/u/5143256?v=3","gravatar_id":"","url":"https://api.github.com/users/hannes-ucsc","html_url":"https://github.com/hannes-ucsc","followers_url":"https://api.github.com/users/hannes-ucsc/followers","following_url":"https://api.github.com/users/hannes-ucsc/following{/other_user}","gists_url":"https://api.github.com/users/hannes-ucsc/gists{/gist_id}","starred_url":"https://api.github.com/users/hannes-ucsc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hannes-ucsc/subscriptions","organizations_url":"https://api.github.com/users/hannes-ucsc/orgs","repos_url":"https://api.github.com/users/hannes-ucsc/repos","events_url":"https://api.github.com/users/hannes-ucsc/events{/privacy}","received_events_url":"https://api.github.com/users/hannes-ucsc/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2015-12-03T19:46:54Z","updated_at":"2015-12-03T19:50:42Z","closed_at":null,"body":"_From @dgaston on December 3, 2015 17:11_\n\nSince there is no general help forum for Toil or Toil scripts and it is in heavy development, I thought this would be the best place to ask. I have been working on porting over my existing pipeline work to Toil due to its broad support for Mesos. I am currently running this pipeline just locally. I wanted to do something a little different because I re-use many of the same methods in different pipelines so all of the methods live in their own project and are then imported into the pipeline script where I construct the workflow and graph using wrapJobFn() to define all of the jobs and then use the addJob() method to chain them together for maximum flexibility. You can see this in this repo: https://github.com/dgaston/ddbio-ngsflow.\r\n\r\nAll of my wrapper methods are in ngsflow and the workflow-* scripts are the various pipelines. workflow-somatic-amplicon.py is the current one I am testing.\r\n\r\nThe problem I am having is that the jobs don't seem to be respecting the resource allocations. For instance I am specifying --maxCores of 4, in my configuration input I specify that BWA should use 4 cores and so that is passed to the cores keyword argument in wrapJobFn(). However when executing I have four different BWA jobs, each multithreaded to 4 threads running (4 different samples) as well as two other jobs executing. It was my understanding that in this case there should only be a single BWA job executing at a time, as each BWA job should be requesting 4 cores out of a max of 4 cores allowed to toil. I see similar issues with memory usage.\r\n\r\nAny help would be appreciated.\n\n_Copied from original issue: BD2KGenomics/toil-scripts#60_"},"comment":{"url":"https://api.github.com/repos/BD2KGenomics/toil/issues/comments/161762920","html_url":"https://github.com/BD2KGenomics/toil/issues/598#issuecomment-161762920","issue_url":"https://api.github.com/repos/BD2KGenomics/toil/issues/598","id":161762920,"user":{"login":"hannes-ucsc","id":5143256,"avatar_url":"https://avatars.githubusercontent.com/u/5143256?v=3","gravatar_id":"","url":"https://api.github.com/users/hannes-ucsc","html_url":"https://github.com/hannes-ucsc","followers_url":"https://api.github.com/users/hannes-ucsc/followers","following_url":"https://api.github.com/users/hannes-ucsc/following{/other_user}","gists_url":"https://api.github.com/users/hannes-ucsc/gists{/gist_id}","starred_url":"https://api.github.com/users/hannes-ucsc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hannes-ucsc/subscriptions","organizations_url":"https://api.github.com/users/hannes-ucsc/orgs","repos_url":"https://api.github.com/users/hannes-ucsc/repos","events_url":"https://api.github.com/users/hannes-ucsc/events{/privacy}","received_events_url":"https://api.github.com/users/hannes-ucsc/received_events","type":"User","site_admin":false},"created_at":"2015-12-03T19:50:42Z","updated_at":"2015-12-03T19:50:42Z","body":"Hi Daniel,\r\n\r\nIf you are using Toil 3.0.7, you are probably running into #435. We just released 3.1.0 with a fix for that issue. Could you try with that version?"}}