{"action":"created","issue":{"url":"https://api.github.com/repos/uswitch/bifrost/issues/10","labels_url":"https://api.github.com/repos/uswitch/bifrost/issues/10/labels{/name}","comments_url":"https://api.github.com/repos/uswitch/bifrost/issues/10/comments","events_url":"https://api.github.com/repos/uswitch/bifrost/issues/10/events","html_url":"https://github.com/uswitch/bifrost/pull/10","id":69531464,"number":10,"title":"Implement support for tar file format.","user":{"login":"tgk","id":95587,"avatar_url":"https://avatars.githubusercontent.com/u/95587?v=3","gravatar_id":"","url":"https://api.github.com/users/tgk","html_url":"https://github.com/tgk","followers_url":"https://api.github.com/users/tgk/followers","following_url":"https://api.github.com/users/tgk/following{/other_user}","gists_url":"https://api.github.com/users/tgk/gists{/gist_id}","starred_url":"https://api.github.com/users/tgk/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tgk/subscriptions","organizations_url":"https://api.github.com/users/tgk/orgs","repos_url":"https://api.github.com/users/tgk/repos","events_url":"https://api.github.com/users/tgk/events{/privacy}","received_events_url":"https://api.github.com/users/tgk/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2015-04-20T08:02:46Z","updated_at":"2015-04-22T12:19:38Z","closed_at":"2015-04-22T12:19:38Z","pull_request":{"url":"https://api.github.com/repos/uswitch/bifrost/pulls/10","html_url":"https://github.com/uswitch/bifrost/pull/10","diff_url":"https://github.com/uswitch/bifrost/pull/10.diff","patch_url":"https://github.com/uswitch/bifrost/pull/10.patch"},"body":"I have added support for tar files as a sequence format. I like adding the option as it is a lower barrier into using Bifrost, and as tar files have excellent tooling on *nix systems.\r\n\r\nThere are some cons though:\r\n\r\n- The change relies on the Apache Commons Compress project, adding yet another dependency to the binary.\r\n- The codebase has grown. A configuration parameter is now passed through countless calls.\r\n\r\nI think the latter point is the most annoying. The call arg lists are waaaaay to long. I wonder if some of the args would scan better moved to records in a more (dare I say) OO like design."},"comment":{"url":"https://api.github.com/repos/uswitch/bifrost/issues/comments/95154685","html_url":"https://github.com/uswitch/bifrost/pull/10#issuecomment-95154685","issue_url":"https://api.github.com/repos/uswitch/bifrost/issues/10","id":95154685,"user":{"login":"tgk","id":95587,"avatar_url":"https://avatars.githubusercontent.com/u/95587?v=3","gravatar_id":"","url":"https://api.github.com/users/tgk","html_url":"https://github.com/tgk","followers_url":"https://api.github.com/users/tgk/followers","following_url":"https://api.github.com/users/tgk/following{/other_user}","gists_url":"https://api.github.com/users/tgk/gists{/gist_id}","starred_url":"https://api.github.com/users/tgk/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tgk/subscriptions","organizations_url":"https://api.github.com/users/tgk/orgs","repos_url":"https://api.github.com/users/tgk/repos","events_url":"https://api.github.com/users/tgk/events{/privacy}","received_events_url":"https://api.github.com/users/tgk/received_events","type":"User","site_admin":false},"created_at":"2015-04-22T12:19:38Z","updated_at":"2015-04-22T12:19:38Z","body":"@ragnard after having' thought about this for a while I think I might just withdraw my pull request. My extension built on the assumption that tar files were a sequence format. It is now clear that they are not. The clue is in the name.\r\n\r\nThis means consumers of these tar files would not be able to stream through them in sequence. Kafka is all about lineralizability, and you would lose exactly that packing the data up in a tar. The prize of making the code more complex and adding yet another dependency (and whatever transient stuff it might pull in) is just not worth it.\r\n\r\nI will consider this a lesson in reading up on data formats.\r\n\r\nIf anyone can point me towards a common, open sequence file format I'll be happy to give it a go, but for now I'm sticking with baldr."}}