{"action":"created","issue":{"url":"https://api.github.com/repos/lh3/fermikit/issues/9","labels_url":"https://api.github.com/repos/lh3/fermikit/issues/9/labels{/name}","comments_url":"https://api.github.com/repos/lh3/fermikit/issues/9/comments","events_url":"https://api.github.com/repos/lh3/fermikit/issues/9/events","html_url":"https://github.com/lh3/fermikit/issues/9","id":107815603,"number":9,"title":"High RAM usage on simulated samples","user":{"login":"morgantaschuk","id":1746206,"avatar_url":"https://avatars.githubusercontent.com/u/1746206?v=3","gravatar_id":"","url":"https://api.github.com/users/morgantaschuk","html_url":"https://github.com/morgantaschuk","followers_url":"https://api.github.com/users/morgantaschuk/followers","following_url":"https://api.github.com/users/morgantaschuk/following{/other_user}","gists_url":"https://api.github.com/users/morgantaschuk/gists{/gist_id}","starred_url":"https://api.github.com/users/morgantaschuk/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/morgantaschuk/subscriptions","organizations_url":"https://api.github.com/users/morgantaschuk/orgs","repos_url":"https://api.github.com/users/morgantaschuk/repos","events_url":"https://api.github.com/users/morgantaschuk/events{/privacy}","received_events_url":"https://api.github.com/users/morgantaschuk/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-09-22T22:30:49Z","updated_at":"2015-09-22T23:41:34Z","closed_at":null,"body":"Hi,\r\n\r\nI have 50x simulated 1000g data with ART (http://www.niehs.nih.gov/research/resources/software/biostatistics/art/). I'm trying to run fermikit on this data and our cluster is killing the job when it passes 170G RAM.  Do you have any suggestions for decreasing memory usage?\r\n\r\n```\r\nfermi.kit/fermi2.pl unitig -s3g -t16 -l126 -p art_50x_fermikit \"fermi.kit/seqtk mergepe NA12877_50x_1.fq.gz NA12877_50x_2.fq.gz\" > art_50x_fermikit.mak\r\nmake -f art_50x_fermikit.mak\r\nfermi.kit/run-calling -t16 reference/hg19_random.fa art_50x_fermikit.mag.gz | sh\r\n```\r\n\r\nThe only thing in the log is the following:\r\n\r\n```\r\nbash -c '/u/mtaschuk/git/fermikit/fermi.kit/bfc -s 3g -t 16 <(~/git/fermikit/fermi.kit/seqtk mergepe NA12877_50x_1.fq.gz NA12877_50x_2.fq.gz) <(~/git/fermikit/fermi.kit/seqtk mergepe NA12877_50x_1.fq.gz NA12877_50x_2.fq.gz) 2> art_50x_fermikit.ec.fq.gz.log | gzip -1 > art_50x_fermikit.ec.fq.gz'\r\n```\r\n\r\nI'm using NA12877 vcf from GiaB, converted to fasta reference, and then simulating using ART with the following characteristics:\r\n\r\n* read length: 126\r\n* mean fragment length: 500\r\n* standard deviation: 120\r\n* average coverage: 50x\r\n* paired end\r\n* error model: HiSeq 2500\r\n"},"comment":{"url":"https://api.github.com/repos/lh3/fermikit/issues/comments/142454148","html_url":"https://github.com/lh3/fermikit/issues/9#issuecomment-142454148","issue_url":"https://api.github.com/repos/lh3/fermikit/issues/9","id":142454148,"user":{"login":"lh3","id":480346,"avatar_url":"https://avatars.githubusercontent.com/u/480346?v=3","gravatar_id":"","url":"https://api.github.com/users/lh3","html_url":"https://github.com/lh3","followers_url":"https://api.github.com/users/lh3/followers","following_url":"https://api.github.com/users/lh3/following{/other_user}","gists_url":"https://api.github.com/users/lh3/gists{/gist_id}","starred_url":"https://api.github.com/users/lh3/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lh3/subscriptions","organizations_url":"https://api.github.com/users/lh3/orgs","repos_url":"https://api.github.com/users/lh3/repos","events_url":"https://api.github.com/users/lh3/events{/privacy}","received_events_url":"https://api.github.com/users/lh3/received_events","type":"User","site_admin":false},"created_at":"2015-09-22T23:41:34Z","updated_at":"2015-09-22T23:41:34Z","body":"Simulators usually generate reads with much higher error rate. The peak memory of the error corrector is sensitive to the error rate. This is an issue with fermikit."}}