{"action":"created","issue":{"url":"https://api.github.com/repos/kafkaex/kafka_ex/issues/35","labels_url":"https://api.github.com/repos/kafkaex/kafka_ex/issues/35/labels{/name}","comments_url":"https://api.github.com/repos/kafkaex/kafka_ex/issues/35/comments","events_url":"https://api.github.com/repos/kafkaex/kafka_ex/issues/35/events","html_url":"https://github.com/kafkaex/kafka_ex/pull/35","id":107295181,"number":35,"title":"Support reading snappy-compressed logs","user":{"login":"dantswain","id":463747,"avatar_url":"https://avatars.githubusercontent.com/u/463747?v=3","gravatar_id":"","url":"https://api.github.com/users/dantswain","html_url":"https://github.com/dantswain","followers_url":"https://api.github.com/users/dantswain/followers","following_url":"https://api.github.com/users/dantswain/following{/other_user}","gists_url":"https://api.github.com/users/dantswain/gists{/gist_id}","starred_url":"https://api.github.com/users/dantswain/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dantswain/subscriptions","organizations_url":"https://api.github.com/users/dantswain/orgs","repos_url":"https://api.github.com/users/dantswain/repos","events_url":"https://api.github.com/users/dantswain/events{/privacy}","received_events_url":"https://api.github.com/users/dantswain/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":9,"created_at":"2015-09-18T23:34:42Z","updated_at":"2015-09-20T22:56:28Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/kafkaex/kafka_ex/pulls/35","html_url":"https://github.com/kafkaex/kafka_ex/pull/35","diff_url":"https://github.com/kafkaex/kafka_ex/pull/35.diff","patch_url":"https://github.com/kafkaex/kafka_ex/pull/35.patch"},"body":"This partially addresses #19 - I am only dealing with snappy and I split reading from writing to make a bite-sized change.  I plan to also work on writing.  Adding other compression formats shouldn't be hard once the protocol details are worked out. \r\n\r\nThere's a few caveats here:\r\n\r\n* I used a fork of the Erlang snappy lib; the only change is that the fork 'supports' Erlang R18 (it's just a check in rebar.config).  There is a PR open on the trunk repository and it should be trivial to switch back to the trunk repo once the PR is merged.\r\n\r\n* Because of the way that compression works, if one fetches an offset that is inside a compressed batch of messages, the entire batch is returned.  This is the behavior for the Ruby client, Poseidon.  It wouldn't be _that_ hard to check the offsets of the batch and filter out ones that were not asked for.\r\n\r\n* `KafkaEx.Util` could certainly use some refactoring.  I did not want to inflict my particular style and potentially introduce merge conflicts.  I'd be glad to have a go at it if you are OK with that.\r\n"},"comment":{"url":"https://api.github.com/repos/kafkaex/kafka_ex/issues/comments/141843295","html_url":"https://github.com/kafkaex/kafka_ex/pull/35#issuecomment-141843295","issue_url":"https://api.github.com/repos/kafkaex/kafka_ex/issues/35","id":141843295,"user":{"login":"dantswain","id":463747,"avatar_url":"https://avatars.githubusercontent.com/u/463747?v=3","gravatar_id":"","url":"https://api.github.com/users/dantswain","html_url":"https://github.com/dantswain","followers_url":"https://api.github.com/users/dantswain/followers","following_url":"https://api.github.com/users/dantswain/following{/other_user}","gists_url":"https://api.github.com/users/dantswain/gists{/gist_id}","starred_url":"https://api.github.com/users/dantswain/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dantswain/subscriptions","organizations_url":"https://api.github.com/users/dantswain/orgs","repos_url":"https://api.github.com/users/dantswain/repos","events_url":"https://api.github.com/users/dantswain/events{/privacy}","received_events_url":"https://api.github.com/users/dantswain/received_events","type":"User","site_admin":false},"created_at":"2015-09-20T22:56:28Z","updated_at":"2015-09-20T22:56:28Z","body":"A handful of the integration tests failed on Travis; I got similar failures on one laptop but not on a different one.  I get different test failures on different runs, so seems like something finicky wrt async?"}}