{"action":"created","issue":{"url":"https://api.github.com/repos/druid-io/druid/issues/1093","labels_url":"https://api.github.com/repos/druid-io/druid/issues/1093/labels{/name}","comments_url":"https://api.github.com/repos/druid-io/druid/issues/1093/comments","events_url":"https://api.github.com/repos/druid-io/druid/issues/1093/events","html_url":"https://github.com/druid-io/druid/pull/1093","id":56719600,"number":1093,"title":"Query time lookup","user":{"login":"drcrallen","id":8213081,"avatar_url":"https://avatars.githubusercontent.com/u/8213081?v=3","gravatar_id":"","url":"https://api.github.com/users/drcrallen","html_url":"https://github.com/drcrallen","followers_url":"https://api.github.com/users/drcrallen/followers","following_url":"https://api.github.com/users/drcrallen/following{/other_user}","gists_url":"https://api.github.com/users/drcrallen/gists{/gist_id}","starred_url":"https://api.github.com/users/drcrallen/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/drcrallen/subscriptions","organizations_url":"https://api.github.com/users/drcrallen/orgs","repos_url":"https://api.github.com/users/drcrallen/repos","events_url":"https://api.github.com/users/drcrallen/events{/privacy}","received_events_url":"https://api.github.com/users/drcrallen/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2015-02-05T19:48:56Z","updated_at":"2015-02-05T22:18:19Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/druid-io/druid/pulls/1093","html_url":"https://github.com/druid-io/druid/pull/1093","diff_url":"https://github.com/druid-io/druid/pull/1093.diff","patch_url":"https://github.com/druid-io/druid/pull/1093.patch"},"body":"This is not quite ready to merge, but is definitely at a point where discussion on the solution can be active.\r\n\r\nThis PR includes https://github.com/druid-io/druid/pull/1092\r\n\r\nAdd query time lookups for renames\r\n    * Add ability to explicitly rename with a map passed at query time\r\n    * Add ability to rename using a key/value lookup in a database\r\n    * Add extension which caches renames from a kafka stream\r\n    * Added README.md for the dim-rename extension\r\n\r\n\r\nTODOs:\r\n* Abstract the implementation of the wiring of the rename function such that the rename dim extraction function only takes one parameter which is the \"key\" for a particular set of implementation configurations. For example: a key of \"publisherId\" can be wired to an implementation of \"use a mysql db located at `<<some JDBC URI>>` and use table with key column of XXX and rename column of YYY\". Alternatively a key of \"regionId\" can be wired to an implementation of \"use kafka topic ABC found through Zookeeper XYZ\". It will be up to the query issuer to know which magic key to use to get the results they want."},"comment":{"url":"https://api.github.com/repos/druid-io/druid/issues/comments/73142098","html_url":"https://github.com/druid-io/druid/pull/1093#issuecomment-73142098","issue_url":"https://api.github.com/repos/druid-io/druid/issues/1093","id":73142098,"user":{"login":"cheddar","id":216778,"avatar_url":"https://avatars.githubusercontent.com/u/216778?v=3","gravatar_id":"","url":"https://api.github.com/users/cheddar","html_url":"https://github.com/cheddar","followers_url":"https://api.github.com/users/cheddar/followers","following_url":"https://api.github.com/users/cheddar/following{/other_user}","gists_url":"https://api.github.com/users/cheddar/gists{/gist_id}","starred_url":"https://api.github.com/users/cheddar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cheddar/subscriptions","organizations_url":"https://api.github.com/users/cheddar/orgs","repos_url":"https://api.github.com/users/cheddar/repos","events_url":"https://api.github.com/users/cheddar/events{/privacy}","received_events_url":"https://api.github.com/users/cheddar/received_events","type":"User","site_admin":false},"created_at":"2015-02-05T22:18:19Z","updated_at":"2015-02-05T22:18:19Z","body":"I'm in agreement with @xvrl.  I think this is an awesome first implementation, but would like to see it a little more decoupled.\r\n\r\nI think we can achieve that by making the \"RenameManager\" a bit less tied to Kafka topics and specific implementations.  That is, what if RenameManager was\r\n\r\n```\r\npublic class RenameManager {\r\n  public RenameManager(Map<String, DimRenameFn>) {...}\r\n\r\n  DimRenameFn find(String name)\r\n}\r\n```\r\n\r\nThen at bootstrap time, we could have something that actually loads up dim extractors into a map and passes that in via DI.  The dim extraction function could refer to the \"table\" by name and just do the join through that.\r\n\r\nEach implementation of the extraction functions could have its own logic and just be registered with the RenameManager.  So, JDBC and Kafka would actually be behind the same interface and the question of what topic, etc. would be something pre-determined.\r\n\r\nI can imagine bootstrapping this with a big json object like\r\n\r\n```\r\n{\r\n   \"publisher\": { \"type\": \"kafka\", \"topic\": \"xyz\" ... },\r\n   \"advertiser\": { \"type\": \"jdbc\", \"url\": \"jdbc://blah/blah\" ... }\r\n}\r\n```\r\n\r\nDoes that seem reasonable/make sense/all that good stuff?"}}