{"action":"created","issue":{"url":"https://api.github.com/repos/apache/flink/issues/1255","labels_url":"https://api.github.com/repos/apache/flink/issues/1255/labels{/name}","comments_url":"https://api.github.com/repos/apache/flink/issues/1255/comments","events_url":"https://api.github.com/repos/apache/flink/issues/1255/events","html_url":"https://github.com/apache/flink/pull/1255","id":110925364,"number":1255,"title":"[FLINK-7] [Runtime] Enable Range Partitioner.","user":{"login":"ChengXiangLi","id":4425616,"avatar_url":"https://avatars.githubusercontent.com/u/4425616?v=3","gravatar_id":"","url":"https://api.github.com/users/ChengXiangLi","html_url":"https://github.com/ChengXiangLi","followers_url":"https://api.github.com/users/ChengXiangLi/followers","following_url":"https://api.github.com/users/ChengXiangLi/following{/other_user}","gists_url":"https://api.github.com/users/ChengXiangLi/gists{/gist_id}","starred_url":"https://api.github.com/users/ChengXiangLi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ChengXiangLi/subscriptions","organizations_url":"https://api.github.com/users/ChengXiangLi/orgs","repos_url":"https://api.github.com/users/ChengXiangLi/repos","events_url":"https://api.github.com/users/ChengXiangLi/events{/privacy}","received_events_url":"https://api.github.com/users/ChengXiangLi/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":6,"created_at":"2015-10-12T07:56:38Z","updated_at":"2015-10-21T10:33:13Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/flink/pulls/1255","html_url":"https://github.com/apache/flink/pull/1255","diff_url":"https://github.com/apache/flink/pull/1255.diff","patch_url":"https://github.com/apache/flink/pull/1255.patch"},"body":"This PR enable range partitioner for Flink follow the path of existing other partitioners. It depends on the sample operator to random sample data from `DataSet` and build range boundaries based on sampled data. 2 other hints about PR:\r\n1. Why execute the sample data job in `JobGraphGenerator` instead of `PartitionOperator`?\r\n     i. launch another job in compile time would lead to infinite job submission, because the `DataSink`s has not been cleared during compile time.\r\n     ii. we need the target stage parallelism to decide sample data size, and `TypeSerializer`/`TypeComparator` to serialize/sort sampled data.\r\n2. Expand the `DataDistribution` API, previous `DataDistribution` take `Key[]` as range boundaries, there is not simple generic way to extract Key from nested object, and `TypeComparator::compareAgainstReference()` is not supported by current comparators. Use `DataSet` elements as the range boundaries make everything much easier, we could use 'TypeComparator::compare()' directly for sort during build `DataDistribution` and selecting channel."},"comment":{"url":"https://api.github.com/repos/apache/flink/issues/comments/149849817","html_url":"https://github.com/apache/flink/pull/1255#issuecomment-149849817","issue_url":"https://api.github.com/repos/apache/flink/issues/1255","id":149849817,"user":{"login":"ChengXiangLi","id":4425616,"avatar_url":"https://avatars.githubusercontent.com/u/4425616?v=3","gravatar_id":"","url":"https://api.github.com/users/ChengXiangLi","html_url":"https://github.com/ChengXiangLi","followers_url":"https://api.github.com/users/ChengXiangLi/followers","following_url":"https://api.github.com/users/ChengXiangLi/following{/other_user}","gists_url":"https://api.github.com/users/ChengXiangLi/gists{/gist_id}","starred_url":"https://api.github.com/users/ChengXiangLi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ChengXiangLi/subscriptions","organizations_url":"https://api.github.com/users/ChengXiangLi/orgs","repos_url":"https://api.github.com/users/ChengXiangLi/repos","events_url":"https://api.github.com/users/ChengXiangLi/events{/privacy}","received_events_url":"https://api.github.com/users/ChengXiangLi/received_events","type":"User","site_admin":false},"created_at":"2015-10-21T10:33:12Z","updated_at":"2015-10-21T10:33:12Z","body":"Implemented the range partitioner with KeySelector based on broadcast data distribution, it's not fully finished yet, @fhueske , for the range partitioner with filed index or field name, it should require the tool you have mentioned before about extracting keys from a record."}}