{"action":"created","issue":{"url":"https://api.github.com/repos/fullcontact/hadoop-sstable/issues/20","labels_url":"https://api.github.com/repos/fullcontact/hadoop-sstable/issues/20/labels{/name}","comments_url":"https://api.github.com/repos/fullcontact/hadoop-sstable/issues/20/comments","events_url":"https://api.github.com/repos/fullcontact/hadoop-sstable/issues/20/events","html_url":"https://github.com/fullcontact/hadoop-sstable/issues/20","id":77719552,"number":20,"title":"Problems with readIndex(FileSystem, Path) method of SSTableIndexIndex","user":{"login":"cfstout","id":4613410,"avatar_url":"https://avatars.githubusercontent.com/u/4613410?v=3","gravatar_id":"","url":"https://api.github.com/users/cfstout","html_url":"https://github.com/cfstout","followers_url":"https://api.github.com/users/cfstout/followers","following_url":"https://api.github.com/users/cfstout/following{/other_user}","gists_url":"https://api.github.com/users/cfstout/gists{/gist_id}","starred_url":"https://api.github.com/users/cfstout/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cfstout/subscriptions","organizations_url":"https://api.github.com/users/cfstout/orgs","repos_url":"https://api.github.com/users/cfstout/repos","events_url":"https://api.github.com/users/cfstout/events{/privacy}","received_events_url":"https://api.github.com/users/cfstout/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2015-05-18T16:34:17Z","updated_at":"2015-05-20T02:28:27Z","closed_at":null,"body":"I have been using this code to create a MR job to run on AWS's elastic map reduce framework, and it seems that there might be a bug in the `readIndex(final FileSystem fileSystem, final Path sstablePath)` method. When we open the index using the nativeS3FileSystem, whenever we call `inputStream.available()` the response is 0. I think the problem is due to the implementation of these inputStream objects, and not necessarily a problem with this repo's code itself. I have managed to fix the issue by moving the code into a while(true) loop and breaking on an EOFException, which though very hacky seems to work. \r\n\r\nI'm not sure if there is a better solution to the problem, or if it's really an artifact of a bug upstream, but thought I'd mention it here so others are aware."},"comment":{"url":"https://api.github.com/repos/fullcontact/hadoop-sstable/issues/comments/103730142","html_url":"https://github.com/fullcontact/hadoop-sstable/issues/20#issuecomment-103730142","issue_url":"https://api.github.com/repos/fullcontact/hadoop-sstable/issues/20","id":103730142,"user":{"login":"bvanberg","id":630897,"avatar_url":"https://avatars.githubusercontent.com/u/630897?v=3","gravatar_id":"","url":"https://api.github.com/users/bvanberg","html_url":"https://github.com/bvanberg","followers_url":"https://api.github.com/users/bvanberg/followers","following_url":"https://api.github.com/users/bvanberg/following{/other_user}","gists_url":"https://api.github.com/users/bvanberg/gists{/gist_id}","starred_url":"https://api.github.com/users/bvanberg/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bvanberg/subscriptions","organizations_url":"https://api.github.com/users/bvanberg/orgs","repos_url":"https://api.github.com/users/bvanberg/repos","events_url":"https://api.github.com/users/bvanberg/events{/privacy}","received_events_url":"https://api.github.com/users/bvanberg/received_events","type":"User","site_admin":false},"created_at":"2015-05-20T02:28:27Z","updated_at":"2015-05-20T02:28:27Z","body":"We have a working branch for 2.0.9 internally. The sstable format changed enough from 1.2 that it required us to change how we're parsing and reading the sstables. I'll make sure we have the latest committed here for others to use. "}}