{"action":"created","issue":{"url":"https://api.github.com/repos/minillinim/GroopM/issues/8","labels_url":"https://api.github.com/repos/minillinim/GroopM/issues/8/labels{/name}","comments_url":"https://api.github.com/repos/minillinim/GroopM/issues/8/comments","events_url":"https://api.github.com/repos/minillinim/GroopM/issues/8/events","html_url":"https://github.com/minillinim/GroopM/issues/8","id":53806257,"number":8,"title":"IndexError while building database using groopm parse","user":{"login":"dburkhardt","id":8322751,"avatar_url":"https://avatars.githubusercontent.com/u/8322751?v=3","gravatar_id":"","url":"https://api.github.com/users/dburkhardt","html_url":"https://github.com/dburkhardt","followers_url":"https://api.github.com/users/dburkhardt/followers","following_url":"https://api.github.com/users/dburkhardt/following{/other_user}","gists_url":"https://api.github.com/users/dburkhardt/gists{/gist_id}","starred_url":"https://api.github.com/users/dburkhardt/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dburkhardt/subscriptions","organizations_url":"https://api.github.com/users/dburkhardt/orgs","repos_url":"https://api.github.com/users/dburkhardt/repos","events_url":"https://api.github.com/users/dburkhardt/events{/privacy}","received_events_url":"https://api.github.com/users/dburkhardt/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2015-01-08T21:56:37Z","updated_at":"2015-01-09T15:39:01Z","closed_at":null,"body":"First off, thanks for working on this, I'm super excited to use this on some soil metagenome data I've been playing with.\r\n\r\nI started to parse three mapping files and build a database, and I encountered an index error. Is it because my bam files are too large? I was surprised this happened even though I only used three files as a test run. The bam files are 1.3G, 8.4G and 1.4G. \r\n\r\nI noticed the following line in my output\r\n>*** Error in `/usr/bin/python': double free or corruption (out): 0x000000005ac1da20 ***\r\n\r\nThis looked familiar because I saw something similar when running nosetests. I've pasted the output from nosetests at the end of this.\r\n\r\n\r\n\r\nGroopM_parse.log:\r\n$ groopm parse test.gm ~/binning_files/1018256.scaffolds.fasta TGACCA.bam TTAGGC.bam TCGGCA.bam\r\n*******************************************************************************\r\n [[GroopM 0.3.3]] Running in data parsing mode...\r\n*******************************************************************************\r\nCreating new database test.gm\r\nParsing contigs\r\nParsing BAM files using 1 threads\r\nParsing file: TGACCA.bam\r\nParsing file: TTAGGC.bam\r\n*** Error in `/usr/bin/python': double free or corruption (out): 0x000000005ac1da20 ***\r\nError creating database: test.gm <type 'exceptions.IndexError'>\r\nUnexpected error: <type 'exceptions.IndexError'>\r\nTraceback (most recent call last):\r\n  File \"/usr/local/bin/groopm\", line 381, in <module>\r\n    GM_parser.parseOptions(args)\r\n  File \"/usr/local/lib/python2.7/dist-packages/groopm/groopm.py\", line 117, in parseOptions\r\n    threads=options.threads)\r\n  File \"/usr/local/lib/python2.7/dist-packages/groopm/mstore.py\", line 276, in createDB\r\n    threads)\r\n  File \"/usr/local/lib/python2.7/dist-packages/groopm/mstore.py\", line 1773, in parse\r\n    return ([BP.BFI.bamFiles[i].fileName for i in range(len(bamFiles))],\r\nIndexError: list index out of range\r\n---------------------------------------------------------------\r\nNosetests.log:\r\ndan@computobacter:~/software/BamM$ nosetests\r\n.*** Error in `/usr/bin/python': double free or corruption (fasttop): 0x0000000001b80bc0 ***\r\n*** Error in `/usr/bin/python': double free or corruption (fasttop): 0x0000000001b80bc0 ***\r\n*** Error in `/usr/bin/python': double free or corruption (fasttop): 0x0000000001b80bc0 ***\r\n*** Error in `/usr/bin/python': double free or corruption (fasttop): 0x0000000001bb5bc0 ***\r\n*** Error in `/usr/bin/python': double free or corruption (fasttop): 0x0000000001bb5bc0 ***\r\n*** Error in `/usr/bin/python': double free or corruption (fasttop): 0x0000000001bb5bc0 ***\r\n*** Error in `/usr/bin/python': double free or corruption (fasttop): 0x00000000017c2bc0 ***\r\n*** Error in `/usr/bin/python': double free or corruption (fasttop): 0x00000000017c2bc0 ***\r\n*** Error in `/usr/bin/python': double free or corruption (fasttop): 0x00000000017c2bc0 ***\r\n*** Error in `/usr/bin/python': double free or corruption (fasttop): 0x000000000132ebc0 ***\r\n*** Error in `/usr/bin/python': double free or corruption (fasttop): 0x000000000132ebc0 ***\r\n*** Error in `/usr/bin/python': double free or corruption (fasttop): 0x000000000132ebc0 ***\r\n*** Error in `/usr/bin/python': double free or corruption (fasttop): 0x00000000025e1060 ***\r\n*** Error in `/usr/bin/python': double free or corruption (fasttop): 0x00000000025e1060 ***\r\n*** Error in `/usr/bin/python': double free or corruption (fasttop): 0x00000000025e1060 ***\r\n*** Error in `/usr/bin/python': double free or corruption (fasttop): 0x00000000012f6bc0 ***\r\n*** Error in `/usr/bin/python': double free or corruption (fasttop): 0x00000000012f6bc0 ***\r\n*** Error in `/usr/bin/python': double free or corruption (fasttop): 0x00000000012f6bc0 ***\r\n............................................\r\n----------------------------------------------------------------------\r\nRan 45 tests in 7.338s\r\n\r\nOK"},"comment":{"url":"https://api.github.com/repos/minillinim/GroopM/issues/comments/69350361","html_url":"https://github.com/minillinim/GroopM/issues/8#issuecomment-69350361","issue_url":"https://api.github.com/repos/minillinim/GroopM/issues/8","id":69350361,"user":{"login":"dburkhardt","id":8322751,"avatar_url":"https://avatars.githubusercontent.com/u/8322751?v=3","gravatar_id":"","url":"https://api.github.com/users/dburkhardt","html_url":"https://github.com/dburkhardt","followers_url":"https://api.github.com/users/dburkhardt/followers","following_url":"https://api.github.com/users/dburkhardt/following{/other_user}","gists_url":"https://api.github.com/users/dburkhardt/gists{/gist_id}","starred_url":"https://api.github.com/users/dburkhardt/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dburkhardt/subscriptions","organizations_url":"https://api.github.com/users/dburkhardt/orgs","repos_url":"https://api.github.com/users/dburkhardt/repos","events_url":"https://api.github.com/users/dburkhardt/events{/privacy}","received_events_url":"https://api.github.com/users/dburkhardt/received_events","type":"User","site_admin":false},"created_at":"2015-01-09T15:39:01Z","updated_at":"2015-01-09T15:39:01Z","body":"I still get the same nosetests errors. I pulled, reran setup.py install, and then ran nosetests. Heres the output and thanks again for your help:\r\n\r\n$ git pull\r\nremote: Counting objects: 10, done.\r\nremote: Compressing objects: 100% (10/10), done.\r\nremote: Total 10 (delta 0), reused 0 (delta 0)\r\nUnpacking objects: 100% (10/10), done.\r\nFrom https://github.com/minillinim/BamM\r\n   7c942a1..8fa33a4  master     -> origin/master\r\nUpdating 7c942a1..8fa33a4\r\nFast-forward\r\n bin/bamm         |   2 +-\r\n c/bamExtractor.c |   6 ++++--\r\n c/bamParser.c    | 136 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-----------------------------\r\n c/bamRead.c      |  22 ++++++++++++++--------\r\n c/pairedLink.c   |  57 +++++++++++++++++++++++++++++++++++++++++++++------------\r\n setup.py         |   2 +-\r\n 6 files changed, 172 insertions(+), 53 deletions(-)\r\n$ sudo python setup.py install\r\n[sudo] password for dan:\r\nchecking for gcc... gcc\r\nchecking whether the C compiler works... yes\r\nchecking for C compiler default output file name... a.out\r\nchecking for suffix of executables...\r\nchecking whether we are cross compiling... no\r\nchecking for suffix of object files... o\r\nchecking whether we are using the GNU C compiler... yes\r\nchecking whether gcc accepts -g... yes\r\nchecking for gcc option to accept ISO C89... none needed\r\nchecking for g++... g++\r\nchecking whether we are using the GNU C++ compiler... yes\r\nchecking whether g++ accepts -g... yes\r\nchecking for lgamma,log,exp in -lm... yes\r\nchecking for libcfu headers in /usr/local/include... found\r\nchecking for libcfu libraries in /usr/local/lib... found\r\nchecking for libhts headers in /usr/local/include... found\r\nchecking for libhts libraries in /usr/local/lib... found\r\nconfigure: creating ./config.status\r\nconfig.status: creating Makefile\r\nconfig.status: creating config.h\r\nconfig.status: config.h is unchanged\r\nrm -f bamParser\r\nrm -f bamExtractor\r\nrm -f *.o\r\nrm -f libBamM.a\r\ngcc -g -fPIC -pthread -O2 -Wall -I/usr/local/include -I/usr/local/include  -c -o bamExtractor.o bamExtractor.c\r\ngcc -g -fPIC -pthread -O2 -Wall -I/usr/local/include -I/usr/local/include  -c -o bamParser.o bamParser.c\r\ngcc -g -fPIC -pthread -O2 -Wall -I/usr/local/include -I/usr/local/include  -c -o pairedLink.o pairedLink.c\r\ngcc -g -fPIC -pthread -O2 -Wall -I/usr/local/include -I/usr/local/include  -c -o bamRead.o bamRead.c\r\ngcc -g -fPIC -pthread -O2 -Wall -I/usr/local/include -I/usr/local/include  -c -o coverageEstimators.o coverageEstimators.c\r\ngcc -g -fPIC -pthread -O2 -Wall -I/usr/local/include -I/usr/local/include -static-libgcc -shared -Wl,-rpath,/usr/local/lib,-soname,libBamM.so.0 -o libBamM.a bamExtractor.c bamParser.c pairedLink.c bamRead.c coverageEstimators.c -lm -L/usr/local/lib -lcfu -L/usr/local/lib -lhts\r\nBuilding library\r\n/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'install_requires'\r\n  warnings.warn(msg)\r\nrunning install\r\nrunning build\r\nrunning build_py\r\nrunning build_scripts\r\ncopying and adjusting bin/bamm -> build/scripts-2.7\r\nrunning install_lib\r\nrunning install_scripts\r\ncopying build/scripts-2.7/bamm -> /usr/local/bin\r\nchanging mode of /usr/local/bin/bamm to 775\r\nchanging mode of /usr/local/bin/bamFlags to 775\r\nrunning install_data\r\ncopying c/libBamM.a -> /usr/local/lib/python2.7/dist-packages/bamm/c/\r\nrunning install_egg_info\r\nWriting /usr/local/lib/python2.7/dist-packages/BamM-1.3.2.egg-info\r\n$ bamm\r\n\r\n                              ...::: BamM :::...\r\n\r\n                    Working with the BAM, not against it...\r\n\r\n   -------------------------------------------------------------------------\r\n                                  version: 1.3.2\r\n   -------------------------------------------------------------------------\r\n\r\n    bamm make     ->  Make BAM/TAM files (sorted + indexed)\r\n    bamm parse    ->  Get coverage profiles / linking reads / insert types\r\n    bamm extract  ->  Extract reads / headers from BAM files\r\n\r\n    USE: bamm OPTION -h to see detailed options\r\n\r\n$ nosetests\r\n.*** Error in `/usr/bin/python': double free or corruption (fasttop): 0x0000000000fe9bc0 ***\r\n*** Error in `/usr/bin/python': double free or corruption (fasttop): 0x0000000000fe9bc0 ***\r\n*** Error in `/usr/bin/python': double free or corruption (fasttop): 0x0000000000fe9bc0 ***\r\n*** Error in `/usr/bin/python': double free or corruption (fasttop): 0x0000000001fe5bc0 ***\r\n*** Error in `/usr/bin/python': double free or corruption (fasttop): 0x0000000001fe5bc0 ***\r\n*** Error in `/usr/bin/python': double free or corruption (fasttop): 0x0000000001fe5bc0 ***\r\n*** Error in `/usr/bin/python': double free or corruption (fasttop): 0x0000000002017bc0 ***\r\n*** Error in `/usr/bin/python': double free or corruption (fasttop): 0x0000000002017bc0 ***\r\n*** Error in `/usr/bin/python': double free or corruption (fasttop): 0x0000000002017bc0 ***\r\n*** Error in `/usr/bin/python': double free or corruption (fasttop): 0x0000000002c42bc0 ***\r\n*** Error in `/usr/bin/python': double free or corruption (fasttop): 0x0000000002c42bc0 ***\r\n*** Error in `/usr/bin/python': double free or corruption (fasttop): 0x0000000002c42bc0 ***\r\n*** Error in `/usr/bin/python': double free or corruption (fasttop): 0x0000000001e40bc0 ***\r\n*** Error in `/usr/bin/python': double free or corruption (fasttop): 0x0000000001e40bc0 ***\r\n*** Error in `/usr/bin/python': double free or corruption (fasttop): 0x0000000001e40bc0 ***\r\n*** Error in `/usr/bin/python': double free or corruption (fasttop): 0x0000000001aebbc0 ***\r\n*** Error in `/usr/bin/python': double free or corruption (fasttop): 0x0000000001aebbc0 ***\r\n*** Error in `/usr/bin/python': double free or corruption (fasttop): 0x0000000001aebbc0 ***\r\n............................................\r\n----------------------------------------------------------------------\r\nRan 45 tests in 7.999s\r\n\r\nOK\r\n\r\n> On Jan 9, 2015, at 1:53 AM, Michael Imelfort <notifications@github.com> wrote:\r\n> \r\n> Hi,\r\n> \r\n> Your install looks OK. I checked through the code and I've removed some dodgyness with some of the free statements. I don't know if this will fix your problems or not, but thanks for pointing me to a bit of code I can fix. The new version is 1.3.2. could you pull this down and run the nosetests on it please.\r\n> —\r\n> Reply to this email directly or view it on GitHub <https://github.com/minillinim/GroopM/issues/8#issuecomment-69299982>.\r\n> "}}