{"action":"created","issue":{"url":"https://api.github.com/repos/apache/kafka/issues/99","labels_url":"https://api.github.com/repos/apache/kafka/issues/99/labels{/name}","comments_url":"https://api.github.com/repos/apache/kafka/issues/99/comments","events_url":"https://api.github.com/repos/apache/kafka/issues/99/events","html_url":"https://github.com/apache/kafka/pull/99","id":97399231,"number":99,"title":"KAFKA-2366: Copycat","user":{"login":"ewencp","id":47366,"avatar_url":"https://avatars.githubusercontent.com/u/47366?v=3","gravatar_id":"","url":"https://api.github.com/users/ewencp","html_url":"https://github.com/ewencp","followers_url":"https://api.github.com/users/ewencp/followers","following_url":"https://api.github.com/users/ewencp/following{/other_user}","gists_url":"https://api.github.com/users/ewencp/gists{/gist_id}","starred_url":"https://api.github.com/users/ewencp/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ewencp/subscriptions","organizations_url":"https://api.github.com/users/ewencp/orgs","repos_url":"https://api.github.com/users/ewencp/repos","events_url":"https://api.github.com/users/ewencp/events{/privacy}","received_events_url":"https://api.github.com/users/ewencp/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":13,"created_at":"2015-07-27T06:54:51Z","updated_at":"2015-08-12T03:03:14Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/kafka/pulls/99","html_url":"https://github.com/apache/kafka/pull/99","diff_url":"https://github.com/apache/kafka/pull/99.diff","patch_url":"https://github.com/apache/kafka/pull/99.patch"},"body":"This is an initial patch implementing the basics of Copycat for KIP-26.\r\n\r\nThe intent here is to start a review of the key pieces of the core API and get a reasonably functional, baseline, non-distributed implementation of Copycat in place to get things rolling. The current patch has a number of known issues that need to be addressed before a final version:\r\n\r\n* Some build-related issues. Specifically, requires some locally-installed dependencies (see below), ignores checkstyle for the runtime data library because it's lifted from Avro currently and likely won't last in its current form, and some Gradle task dependencies aren't quite right because I haven't gotten rid of the dependency on `core` (which should now be an easy patch since new consumer groups are in a much better state).\r\n* This patch currently depends on some Confluent trunk code because I prototyped with our Avro serializers w/ schema-registry support. We need to figure out what we want to provide as an example built-in set of serializers. Unlike core Kafka where we could ignore the issue, providing only ByteArray or String serializers, this is pretty central to how Copycat works.\r\n* This patch uses a hacked up version of Avro as its runtime data format. Not sure if we want to go through the entire API discussion just to get some basic code committed, so I filed KAFKA-2367 to handle that separately. The core connector APIs and the runtime data APIs are entirely orthogonal.\r\n* This patch needs some updates to get aligned with recent new consumer changes (specifically, I'm aware of the ConcurrentModificationException issue on exit). More generally, the new consumer is in flux but Copycat depends on it, so there are likely to be some negative interactions.\r\n* The layout feels a bit awkward to me right now because I ported it from a Maven layout. We don't have nearly the same level of granularity in Kafka currently (core and clients, plus the mostly ignored examples, log4j-appender, and a couple of contribs). We might want to reorganize, although keeping data+api separate from runtime and connector plugins is useful for minimizing dependencies.\r\n* There are a variety of other things (e.g., I'm not happy with the exception hierarchy/how they are currently handled, TopicPartition doesn't really need to be duplicated unless we want Copycat entirely isolated from the Kafka APIs, etc), but I expect those we'll cover in the review.\r\n\r\nBefore commenting on the patch, it's probably worth reviewing https://issues.apache.org/jira/browse/KAFKA-2365 and https://issues.apache.org/jira/browse/KAFKA-2366 to get an idea of what I had in mind for a) what we ultimately want with all the Copycat patches and b) what we aim to cover in this initial patch. My hope is that we can use a WIP patch (after the current obvious deficiencies are addressed) while recognizing that we want to make iterative progress with a bunch of subsequent PRs."},"comment":{"url":"https://api.github.com/repos/apache/kafka/issues/comments/130142405","html_url":"https://github.com/apache/kafka/pull/99#issuecomment-130142405","issue_url":"https://api.github.com/repos/apache/kafka/issues/99","id":130142405,"user":{"login":"asfbot","id":5243949,"avatar_url":"https://avatars.githubusercontent.com/u/5243949?v=3","gravatar_id":"","url":"https://api.github.com/users/asfbot","html_url":"https://github.com/asfbot","followers_url":"https://api.github.com/users/asfbot/followers","following_url":"https://api.github.com/users/asfbot/following{/other_user}","gists_url":"https://api.github.com/users/asfbot/gists{/gist_id}","starred_url":"https://api.github.com/users/asfbot/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/asfbot/subscriptions","organizations_url":"https://api.github.com/users/asfbot/orgs","repos_url":"https://api.github.com/users/asfbot/repos","events_url":"https://api.github.com/users/asfbot/events{/privacy}","received_events_url":"https://api.github.com/users/asfbot/received_events","type":"User","site_admin":false},"created_at":"2015-08-12T03:03:14Z","updated_at":"2015-08-12T03:03:14Z","body":"[kafka-trunk-git-pr #130](https://builds.apache.org/job/kafka-trunk-git-pr/130/) SUCCESS\nThis pull request looks good"}}