{"action":"created","issue":{"url":"https://api.github.com/repos/arq5x/grabix/issues/21","labels_url":"https://api.github.com/repos/arq5x/grabix/issues/21/labels{/name}","comments_url":"https://api.github.com/repos/arq5x/grabix/issues/21/comments","events_url":"https://api.github.com/repos/arq5x/grabix/issues/21/events","html_url":"https://github.com/arq5x/grabix/issues/21","id":78494618,"number":21,"title":"Possible off-by-1 error at chunksize boundary","user":{"login":"schelhorn","id":5024959,"avatar_url":"https://avatars.githubusercontent.com/u/5024959?v=3","gravatar_id":"","url":"https://api.github.com/users/schelhorn","html_url":"https://github.com/schelhorn","followers_url":"https://api.github.com/users/schelhorn/followers","following_url":"https://api.github.com/users/schelhorn/following{/other_user}","gists_url":"https://api.github.com/users/schelhorn/gists{/gist_id}","starred_url":"https://api.github.com/users/schelhorn/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/schelhorn/subscriptions","organizations_url":"https://api.github.com/users/schelhorn/orgs","repos_url":"https://api.github.com/users/schelhorn/repos","events_url":"https://api.github.com/users/schelhorn/events{/privacy}","received_events_url":"https://api.github.com/users/schelhorn/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2015-05-20T11:55:01Z","updated_at":"2015-05-20T12:57:22Z","closed_at":null,"body":"When working on `grabix` indexed FASTQ files in `bcbio`, I get an off-by-one error since the latest commit six days ago (version 0.1.5) when crossing the chunksize ([which is 10000](https://github.com/arq5x/grabix/blob/5542ecb027b615e10b7f462161b553953d5394af/grabix.h#L15)), possibly due to an earlier off-by-1 fix in 8d81fed that may did not have the intended result (the latter is pure speculation on my side). As a result, I get malformed FASTQ files when grabbing ranges of rows that are multiplies of 4 (the length of a canonical FASTQ record) whenever I perform a grab that starts at more than 10000 lines.\r\n\r\nFor instance, this is a asking for a grab that starts with the first entry of a FASTQ record (the FASTQ ID), since `9997 % 4 == 1` and it works fine since we start below the chunksize:\r\n\r\n```shell\r\n$ grabix grab SAMPLE.fq.gz 9997 10012\r\n@2500/1\r\nCATAGCCATGGCCTTTGACAGATACATGGCCATATGTAAACCTCTCCACTACCTGACCATCATGAGCCCAAGAATGTGTCTATACTTTTT\r\n+\r\nGGEGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGEGGGGGGGGFGGGGGGGGFGGGEGGEGGGGGGEGGGEGGFEGGGGF\r\n@2501/1\r\nCGGTCCGGGGTGGTCCAGTCTGATCCCAACCGGCCCACCCGGGGCATCCGGTGGAAGTCTTCGCCGGAGGATCCGAAGGCAGCATCAACG\r\n+\r\nGGGGGEFFFFBFECFFFFFFGGGGGGGGGGGEFGGGGGGGGGEGG?:EEEECEEEEFEGGG=GEGGGEGGEEFEFBDCDA?DB?DF?EFE\r\n@2502/1\r\nGAGGCTGTGCTGTTTCTCTTTCTCTTCTAGGATATCAAAGTCGTCTTGTGCAGGTATGATGGGCCACCCCAAATGTATTTTGCCTACTGA\r\n+\r\nGGGGGGGGGGGGGGGGGGGGGGFGGGEGDGGGGGGFGGGGBGGGGGGEEGFEFFCEFDEEGEGDGGEGGE=DEEEAGFGGFBFFD=ED=C\r\n@2503/1\r\nAGTCTTATTGTTGGTTTGACTGTGAGTATATGGAACTCTGGCAAAATGAAATGGAGCAAAACACACAAAGAAGACAGCCACGACAACAAA\r\n+\r\nGGGGGGGGGGGGDAGFGBGGGFFGGGEDFEGGGGEGGGGGGGGGGGGGEGGGGDEGGGFDGGGFGGGFDEDGGGGGGGGGGGDGGFGDD=\r\n```\r\n\r\nHowever, when I start the grab on the exact same file and start 4 rows further along at start position ` 10001` `(10001 % 4 == 1)` which should give me the FASTQ ID of the next record as first line, I get:\r\n\r\n```shell\r\n$ grabix grab SAMPLE.fq.gz 10001 1008\r\nCGGTCCGGGGTGGTCCAGTCTGATCCCAACCGGCCCACCCGGGGCATCCGGTGGAAGTCTTCGCCGGAGGATCCGAAGGCAGCATCAACG\r\n+\r\nGGGGGEFFFFBFECFFFFFFGGGGGGGGGGGEFGGGGGGGGGEGG?:EEEECEEEEFEGGG=GEGGGEGGEEFEFBDCDA?DB?DF?EFE\r\n@2502/1\r\nGAGGCTGTGCTGTTTCTCTTTCTCTTCTAGGATATCAAAGTCGTCTTGTGCAGGTATGATGGGCCACCCCAAATGTATTTTGCCTACTGA\r\n+\r\nGGGGGGGGGGGGGGGGGGGGGGFGGGEGDGGGGGGFGGGGBGGGGGGEEGFEFFCEFDEEGEGDGGEGGE=DEEEAGFGGFBFFD=ED=C\r\n@2503/1\r\n```\r\nNote that the latter entry does not start with line `10001` but with line `10002`, and also ends with line `10009` instead of line `10008`. The input FASTQ file can be parsed fine using a hand-made parser that checks for validity. Also, the validity of the input file can be checked here in the first output (see above) that also covers the chunksize boundary and includes all relevant records that are erroneously chunked in the second entry."},"comment":{"url":"https://api.github.com/repos/arq5x/grabix/issues/comments/103875286","html_url":"https://github.com/arq5x/grabix/issues/21#issuecomment-103875286","issue_url":"https://api.github.com/repos/arq5x/grabix/issues/21","id":103875286,"user":{"login":"arq5x","id":72291,"avatar_url":"https://avatars.githubusercontent.com/u/72291?v=3","gravatar_id":"","url":"https://api.github.com/users/arq5x","html_url":"https://github.com/arq5x","followers_url":"https://api.github.com/users/arq5x/followers","following_url":"https://api.github.com/users/arq5x/following{/other_user}","gists_url":"https://api.github.com/users/arq5x/gists{/gist_id}","starred_url":"https://api.github.com/users/arq5x/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/arq5x/subscriptions","organizations_url":"https://api.github.com/users/arq5x/orgs","repos_url":"https://api.github.com/users/arq5x/repos","events_url":"https://api.github.com/users/arq5x/events{/privacy}","received_events_url":"https://api.github.com/users/arq5x/received_events","type":"User","site_admin":false},"created_at":"2015-05-20T12:57:22Z","updated_at":"2015-05-20T12:57:22Z","body":"Thanks for reporting this. I suspect the fix we added the other day for small files had an unexpected consequence. We will try to fix this and add functional tests asap. "}}