{"action":"created","comment":{"url":"https://api.github.com/repos/stripe/brushfire/pulls/comments/44470662","id":44470662,"diff_hunk":"@@ -32,8 +32,17 @@ abstract class CSVJob(args: Args) extends TrainerJob(args) {\n     fn\n   }\n \n-  def train(trainingData: TypedPipe[Instance[String, Double, Map[String, Long]]]): Trainer[String, Double, Map[String, Long]] = {\n-    Trainer(trainingData, KFoldSampler(4))\n+  private def id(m: DefaultMetadata): String = {\n+    m.id\n+  }\n+\n+  def train(trainingData: TypedPipe[Instance[DefaultMetadata, Map[String, Double], Map[String, Long]]]): Trainer[DefaultMetadata, String, Double, Map[String, Long], Unit] = {\n+    val annotator = defaultMetadataAnnotator[Unit]","path":"brushfire-scalding/src/main/scala/com/stripe/brushfire/scalding/Example.scala","position":36,"original_position":36,"commit_id":"9fe0d4e77043445ba519ed67d18cf9acb2868091","original_commit_id":"9fe0d4e77043445ba519ed67d18cf9acb2868091","user":{"login":"NathanHowell","id":170829,"avatar_url":"https://avatars.githubusercontent.com/u/170829?v=3","gravatar_id":"","url":"https://api.github.com/users/NathanHowell","html_url":"https://github.com/NathanHowell","followers_url":"https://api.github.com/users/NathanHowell/followers","following_url":"https://api.github.com/users/NathanHowell/following{/other_user}","gists_url":"https://api.github.com/users/NathanHowell/gists{/gist_id}","starred_url":"https://api.github.com/users/NathanHowell/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/NathanHowell/subscriptions","organizations_url":"https://api.github.com/users/NathanHowell/orgs","repos_url":"https://api.github.com/users/NathanHowell/repos","events_url":"https://api.github.com/users/NathanHowell/events{/privacy}","received_events_url":"https://api.github.com/users/NathanHowell/received_events","type":"User","site_admin":false},"body":"`Annotator` isn't an implicit so it needs to be passed explicitly... so it's not really a \"default\"","created_at":"2015-11-10T21:51:56Z","updated_at":"2015-11-10T21:51:56Z","html_url":"https://github.com/stripe/brushfire/pull/74#discussion_r44470662","pull_request_url":"https://api.github.com/repos/stripe/brushfire/pulls/74","_links":{"self":{"href":"https://api.github.com/repos/stripe/brushfire/pulls/comments/44470662"},"html":{"href":"https://github.com/stripe/brushfire/pull/74#discussion_r44470662"},"pull_request":{"href":"https://api.github.com/repos/stripe/brushfire/pulls/74"}}},"pull_request":{"url":"https://api.github.com/repos/stripe/brushfire/pulls/74","id":47227592,"html_url":"https://github.com/stripe/brushfire/pull/74","diff_url":"https://github.com/stripe/brushfire/pull/74.diff","patch_url":"https://github.com/stripe/brushfire/pull/74.patch","issue_url":"https://api.github.com/repos/stripe/brushfire/issues/74","number":74,"state":"open","locked":false,"title":"Annotated instances WIP","user":{"login":"NathanHowell","id":170829,"avatar_url":"https://avatars.githubusercontent.com/u/170829?v=3","gravatar_id":"","url":"https://api.github.com/users/NathanHowell","html_url":"https://github.com/NathanHowell","followers_url":"https://api.github.com/users/NathanHowell/followers","following_url":"https://api.github.com/users/NathanHowell/following{/other_user}","gists_url":"https://api.github.com/users/NathanHowell/gists{/gist_id}","starred_url":"https://api.github.com/users/NathanHowell/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/NathanHowell/subscriptions","organizations_url":"https://api.github.com/users/NathanHowell/orgs","repos_url":"https://api.github.com/users/NathanHowell/repos","events_url":"https://api.github.com/users/NathanHowell/events{/privacy}","received_events_url":"https://api.github.com/users/NathanHowell/received_events","type":"User","site_admin":false},"body":"NOTE: this doesn't work right now, wanted to open this now before I actually complete the work.\r\n\r\nI'm looking at efficient ways to calculate some statistics from Instance instances and attach them to tree leaf nodes. One idea I had was to replace the timestamp parameter (which I don't always need) with `A` and use `Semigroup.sumOption` to populate `LeafNode` instances as they're created. If the timestamp is needed it may also be an interesting thing to track with a `(Min[Long], Max[Long])` tuple (or something fancier) to see time ranges per node.\r\n\r\nThe real world problem is that I have a lot of `Instance`s and a lot of customers. An `Instance` is tied to a single customer, but a customer can have multiple `Instance`s. I want to count the number of distinct customers per split (via `Monoid[HyperLogLog]`, which is usually much different from the target distribution. It would be interesting to calculate these statistics per target as well.","created_at":"2015-10-09T02:09:16Z","updated_at":"2015-11-10T21:51:56Z","closed_at":null,"merged_at":null,"merge_commit_sha":"3f20e9179e21fd4e1fd2ed5aefec3f72f8153ef6","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/stripe/brushfire/pulls/74/commits","review_comments_url":"https://api.github.com/repos/stripe/brushfire/pulls/74/comments","review_comment_url":"https://api.github.com/repos/stripe/brushfire/pulls/comments{/number}","comments_url":"https://api.github.com/repos/stripe/brushfire/issues/74/comments","statuses_url":"https://api.github.com/repos/stripe/brushfire/statuses/9fe0d4e77043445ba519ed67d18cf9acb2868091","head":{"label":"NathanHowell:instance-annotations","ref":"instance-annotations","sha":"9fe0d4e77043445ba519ed67d18cf9acb2868091","user":{"login":"NathanHowell","id":170829,"avatar_url":"https://avatars.githubusercontent.com/u/170829?v=3","gravatar_id":"","url":"https://api.github.com/users/NathanHowell","html_url":"https://github.com/NathanHowell","followers_url":"https://api.github.com/users/NathanHowell/followers","following_url":"https://api.github.com/users/NathanHowell/following{/other_user}","gists_url":"https://api.github.com/users/NathanHowell/gists{/gist_id}","starred_url":"https://api.github.com/users/NathanHowell/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/NathanHowell/subscriptions","organizations_url":"https://api.github.com/users/NathanHowell/orgs","repos_url":"https://api.github.com/users/NathanHowell/repos","events_url":"https://api.github.com/users/NathanHowell/events{/privacy}","received_events_url":"https://api.github.com/users/NathanHowell/received_events","type":"User","site_admin":false},"repo":{"id":41122734,"name":"brushfire","full_name":"NathanHowell/brushfire","owner":{"login":"NathanHowell","id":170829,"avatar_url":"https://avatars.githubusercontent.com/u/170829?v=3","gravatar_id":"","url":"https://api.github.com/users/NathanHowell","html_url":"https://github.com/NathanHowell","followers_url":"https://api.github.com/users/NathanHowell/followers","following_url":"https://api.github.com/users/NathanHowell/following{/other_user}","gists_url":"https://api.github.com/users/NathanHowell/gists{/gist_id}","starred_url":"https://api.github.com/users/NathanHowell/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/NathanHowell/subscriptions","organizations_url":"https://api.github.com/users/NathanHowell/orgs","repos_url":"https://api.github.com/users/NathanHowell/repos","events_url":"https://api.github.com/users/NathanHowell/events{/privacy}","received_events_url":"https://api.github.com/users/NathanHowell/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/NathanHowell/brushfire","description":"Distributed decision tree ensemble learning in Scala","fork":true,"url":"https://api.github.com/repos/NathanHowell/brushfire","forks_url":"https://api.github.com/repos/NathanHowell/brushfire/forks","keys_url":"https://api.github.com/repos/NathanHowell/brushfire/keys{/key_id}","collaborators_url":"https://api.github.com/repos/NathanHowell/brushfire/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/NathanHowell/brushfire/teams","hooks_url":"https://api.github.com/repos/NathanHowell/brushfire/hooks","issue_events_url":"https://api.github.com/repos/NathanHowell/brushfire/issues/events{/number}","events_url":"https://api.github.com/repos/NathanHowell/brushfire/events","assignees_url":"https://api.github.com/repos/NathanHowell/brushfire/assignees{/user}","branches_url":"https://api.github.com/repos/NathanHowell/brushfire/branches{/branch}","tags_url":"https://api.github.com/repos/NathanHowell/brushfire/tags","blobs_url":"https://api.github.com/repos/NathanHowell/brushfire/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/NathanHowell/brushfire/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/NathanHowell/brushfire/git/refs{/sha}","trees_url":"https://api.github.com/repos/NathanHowell/brushfire/git/trees{/sha}","statuses_url":"https://api.github.com/repos/NathanHowell/brushfire/statuses/{sha}","languages_url":"https://api.github.com/repos/NathanHowell/brushfire/languages","stargazers_url":"https://api.github.com/repos/NathanHowell/brushfire/stargazers","contributors_url":"https://api.github.com/repos/NathanHowell/brushfire/contributors","subscribers_url":"https://api.github.com/repos/NathanHowell/brushfire/subscribers","subscription_url":"https://api.github.com/repos/NathanHowell/brushfire/subscription","commits_url":"https://api.github.com/repos/NathanHowell/brushfire/commits{/sha}","git_commits_url":"https://api.github.com/repos/NathanHowell/brushfire/git/commits{/sha}","comments_url":"https://api.github.com/repos/NathanHowell/brushfire/comments{/number}","issue_comment_url":"https://api.github.com/repos/NathanHowell/brushfire/issues/comments{/number}","contents_url":"https://api.github.com/repos/NathanHowell/brushfire/contents/{+path}","compare_url":"https://api.github.com/repos/NathanHowell/brushfire/compare/{base}...{head}","merges_url":"https://api.github.com/repos/NathanHowell/brushfire/merges","archive_url":"https://api.github.com/repos/NathanHowell/brushfire/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/NathanHowell/brushfire/downloads","issues_url":"https://api.github.com/repos/NathanHowell/brushfire/issues{/number}","pulls_url":"https://api.github.com/repos/NathanHowell/brushfire/pulls{/number}","milestones_url":"https://api.github.com/repos/NathanHowell/brushfire/milestones{/number}","notifications_url":"https://api.github.com/repos/NathanHowell/brushfire/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/NathanHowell/brushfire/labels{/name}","releases_url":"https://api.github.com/repos/NathanHowell/brushfire/releases{/id}","created_at":"2015-08-20T23:03:16Z","updated_at":"2015-08-20T23:03:17Z","pushed_at":"2015-11-10T21:42:36Z","git_url":"git://github.com/NathanHowell/brushfire.git","ssh_url":"git@github.com:NathanHowell/brushfire.git","clone_url":"https://github.com/NathanHowell/brushfire.git","svn_url":"https://github.com/NathanHowell/brushfire","homepage":null,"size":1461,"stargazers_count":0,"watchers_count":0,"language":"Scala","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":true,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"stripe:master","ref":"master","sha":"74b04a9de59ce5d6a26996867d1cac4617af3a63","user":{"login":"stripe","id":856813,"avatar_url":"https://avatars.githubusercontent.com/u/856813?v=3","gravatar_id":"","url":"https://api.github.com/users/stripe","html_url":"https://github.com/stripe","followers_url":"https://api.github.com/users/stripe/followers","following_url":"https://api.github.com/users/stripe/following{/other_user}","gists_url":"https://api.github.com/users/stripe/gists{/gist_id}","starred_url":"https://api.github.com/users/stripe/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/stripe/subscriptions","organizations_url":"https://api.github.com/users/stripe/orgs","repos_url":"https://api.github.com/users/stripe/repos","events_url":"https://api.github.com/users/stripe/events{/privacy}","received_events_url":"https://api.github.com/users/stripe/received_events","type":"Organization","site_admin":false},"repo":{"id":26926567,"name":"brushfire","full_name":"stripe/brushfire","owner":{"login":"stripe","id":856813,"avatar_url":"https://avatars.githubusercontent.com/u/856813?v=3","gravatar_id":"","url":"https://api.github.com/users/stripe","html_url":"https://github.com/stripe","followers_url":"https://api.github.com/users/stripe/followers","following_url":"https://api.github.com/users/stripe/following{/other_user}","gists_url":"https://api.github.com/users/stripe/gists{/gist_id}","starred_url":"https://api.github.com/users/stripe/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/stripe/subscriptions","organizations_url":"https://api.github.com/users/stripe/orgs","repos_url":"https://api.github.com/users/stripe/repos","events_url":"https://api.github.com/users/stripe/events{/privacy}","received_events_url":"https://api.github.com/users/stripe/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/stripe/brushfire","description":"Distributed decision tree ensemble learning in Scala","fork":false,"url":"https://api.github.com/repos/stripe/brushfire","forks_url":"https://api.github.com/repos/stripe/brushfire/forks","keys_url":"https://api.github.com/repos/stripe/brushfire/keys{/key_id}","collaborators_url":"https://api.github.com/repos/stripe/brushfire/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/stripe/brushfire/teams","hooks_url":"https://api.github.com/repos/stripe/brushfire/hooks","issue_events_url":"https://api.github.com/repos/stripe/brushfire/issues/events{/number}","events_url":"https://api.github.com/repos/stripe/brushfire/events","assignees_url":"https://api.github.com/repos/stripe/brushfire/assignees{/user}","branches_url":"https://api.github.com/repos/stripe/brushfire/branches{/branch}","tags_url":"https://api.github.com/repos/stripe/brushfire/tags","blobs_url":"https://api.github.com/repos/stripe/brushfire/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/stripe/brushfire/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/stripe/brushfire/git/refs{/sha}","trees_url":"https://api.github.com/repos/stripe/brushfire/git/trees{/sha}","statuses_url":"https://api.github.com/repos/stripe/brushfire/statuses/{sha}","languages_url":"https://api.github.com/repos/stripe/brushfire/languages","stargazers_url":"https://api.github.com/repos/stripe/brushfire/stargazers","contributors_url":"https://api.github.com/repos/stripe/brushfire/contributors","subscribers_url":"https://api.github.com/repos/stripe/brushfire/subscribers","subscription_url":"https://api.github.com/repos/stripe/brushfire/subscription","commits_url":"https://api.github.com/repos/stripe/brushfire/commits{/sha}","git_commits_url":"https://api.github.com/repos/stripe/brushfire/git/commits{/sha}","comments_url":"https://api.github.com/repos/stripe/brushfire/comments{/number}","issue_comment_url":"https://api.github.com/repos/stripe/brushfire/issues/comments{/number}","contents_url":"https://api.github.com/repos/stripe/brushfire/contents/{+path}","compare_url":"https://api.github.com/repos/stripe/brushfire/compare/{base}...{head}","merges_url":"https://api.github.com/repos/stripe/brushfire/merges","archive_url":"https://api.github.com/repos/stripe/brushfire/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/stripe/brushfire/downloads","issues_url":"https://api.github.com/repos/stripe/brushfire/issues{/number}","pulls_url":"https://api.github.com/repos/stripe/brushfire/pulls{/number}","milestones_url":"https://api.github.com/repos/stripe/brushfire/milestones{/number}","notifications_url":"https://api.github.com/repos/stripe/brushfire/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/stripe/brushfire/labels{/name}","releases_url":"https://api.github.com/repos/stripe/brushfire/releases{/id}","created_at":"2014-11-20T18:47:07Z","updated_at":"2015-11-07T16:45:42Z","pushed_at":"2015-11-10T21:42:37Z","git_url":"git://github.com/stripe/brushfire.git","ssh_url":"git@github.com:stripe/brushfire.git","clone_url":"https://github.com/stripe/brushfire.git","svn_url":"https://github.com/stripe/brushfire","homepage":null,"size":2185,"stargazers_count":277,"watchers_count":277,"language":"Scala","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":true,"forks_count":26,"mirror_url":null,"open_issues_count":25,"forks":26,"open_issues":25,"watchers":277,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/stripe/brushfire/pulls/74"},"html":{"href":"https://github.com/stripe/brushfire/pull/74"},"issue":{"href":"https://api.github.com/repos/stripe/brushfire/issues/74"},"comments":{"href":"https://api.github.com/repos/stripe/brushfire/issues/74/comments"},"review_comments":{"href":"https://api.github.com/repos/stripe/brushfire/pulls/74/comments"},"review_comment":{"href":"https://api.github.com/repos/stripe/brushfire/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/stripe/brushfire/pulls/74/commits"},"statuses":{"href":"https://api.github.com/repos/stripe/brushfire/statuses/9fe0d4e77043445ba519ed67d18cf9acb2868091"}}}}