{"action":"created","issue":{"url":"https://api.github.com/repos/chapmanb/bcbio-nextgen/issues/1013","labels_url":"https://api.github.com/repos/chapmanb/bcbio-nextgen/issues/1013/labels{/name}","comments_url":"https://api.github.com/repos/chapmanb/bcbio-nextgen/issues/1013/comments","events_url":"https://api.github.com/repos/chapmanb/bcbio-nextgen/issues/1013/events","html_url":"https://github.com/chapmanb/bcbio-nextgen/issues/1013","id":106308635,"number":1013,"title":"featureCounts name-sorts BAM input file in current working directory with a single core","user":{"login":"schelhorn","id":5024959,"avatar_url":"https://avatars.githubusercontent.com/u/5024959?v=3","gravatar_id":"","url":"https://api.github.com/users/schelhorn","html_url":"https://github.com/schelhorn","followers_url":"https://api.github.com/users/schelhorn/followers","following_url":"https://api.github.com/users/schelhorn/following{/other_user}","gists_url":"https://api.github.com/users/schelhorn/gists{/gist_id}","starred_url":"https://api.github.com/users/schelhorn/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/schelhorn/subscriptions","organizations_url":"https://api.github.com/users/schelhorn/orgs","repos_url":"https://api.github.com/users/schelhorn/repos","events_url":"https://api.github.com/users/schelhorn/events{/privacy}","received_events_url":"https://api.github.com/users/schelhorn/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2015-09-14T09:54:30Z","updated_at":"2015-09-14T11:05:12Z","closed_at":null,"body":"Usually, in the `rnaseq` workflow, quantitation for paired inputs with `featureCounts` runs on coordinate-sorted BAM inputs (since, for instance `cufflinks` requires it to be). Due to internal requirements of `featureCounts` (ie., since only allows for feature hits where both read ends align), these coordinate-sorted BAM input files are detected by the `featureCounts` binary and name-sorted prior to processing. Name-sorting occurs by `featureCounts` itself (as part of the binary), on a single core and in the current working directory. \r\n\r\nAs a consequence, that sorting happens in the project working directory of bcbio that usually resides on a shared file system. As a result, a lot of single-core processing power is wasted on sorting these BAMs and the shared file system breaks down if you run a couple of hundred samples in parallel. \r\n\r\nIt would be great if instead the bcbio featurecounts.py module could name-sort the inputs before calling `featureCounts` using the appropriate bcbio function so that sorting happens on local disk and with the specified number of `samtools` cores. `featureCounts` would then (hopefully) refrain from sorting it again. \r\n\r\nNote that the name-sorted BAM file is only used by `featureCounts` anyway (as far as I know), so that file could stay on local disk and be removed after `featureCounts` finishes. In particular, there is no need to copy it over to the shared file system at all (technically, sorting could just be run in the `featureCounts` execution context on the same node `featureCounts` is ran, but one would require `samtools` to be in the parallel environment resource specs, of course)."},"comment":{"url":"https://api.github.com/repos/chapmanb/bcbio-nextgen/issues/comments/140040692","html_url":"https://github.com/chapmanb/bcbio-nextgen/issues/1013#issuecomment-140040692","issue_url":"https://api.github.com/repos/chapmanb/bcbio-nextgen/issues/1013","id":140040692,"user":{"login":"schelhorn","id":5024959,"avatar_url":"https://avatars.githubusercontent.com/u/5024959?v=3","gravatar_id":"","url":"https://api.github.com/users/schelhorn","html_url":"https://github.com/schelhorn","followers_url":"https://api.github.com/users/schelhorn/followers","following_url":"https://api.github.com/users/schelhorn/following{/other_user}","gists_url":"https://api.github.com/users/schelhorn/gists{/gist_id}","starred_url":"https://api.github.com/users/schelhorn/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/schelhorn/subscriptions","organizations_url":"https://api.github.com/users/schelhorn/orgs","repos_url":"https://api.github.com/users/schelhorn/repos","events_url":"https://api.github.com/users/schelhorn/events{/privacy}","received_events_url":"https://api.github.com/users/schelhorn/received_events","type":"User","site_admin":false},"created_at":"2015-09-14T11:05:12Z","updated_at":"2015-09-14T11:05:12Z","body":"@mjafin, true and certainly preferred from my end, although one could argue that the additional filtering options available to `featureCounts` (such as only counting when both mate ends align to the feature) are also nice to have. Still, I'd prefer the STAR way here, too. Also, there is a second `STAR` `--quantMode` option, `TranscriptomeSAM`. Both quantitations can be run in parallel, so we could also do away with the separate `bowtie2` transcriptome alignment for `express` (also allows for direct BAM output and RSEM compatibility). @roryk, you're managing the `rnaseq` workflow, what is your thinking in this regard?"}}