{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/7294","labels_url":"https://api.github.com/repos/apache/spark/issues/7294/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/7294/comments","events_url":"https://api.github.com/repos/apache/spark/issues/7294/events","html_url":"https://github.com/apache/spark/pull/7294","id":93863738,"number":7294,"title":"[SPARK-7131] [ml] Copy Decision Tree, Random Forest impl to spark.ml","user":{"login":"jkbradley","id":5084283,"avatar_url":"https://avatars.githubusercontent.com/u/5084283?v=3","gravatar_id":"","url":"https://api.github.com/users/jkbradley","html_url":"https://github.com/jkbradley","followers_url":"https://api.github.com/users/jkbradley/followers","following_url":"https://api.github.com/users/jkbradley/following{/other_user}","gists_url":"https://api.github.com/users/jkbradley/gists{/gist_id}","starred_url":"https://api.github.com/users/jkbradley/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jkbradley/subscriptions","organizations_url":"https://api.github.com/users/jkbradley/orgs","repos_url":"https://api.github.com/users/jkbradley/repos","events_url":"https://api.github.com/users/jkbradley/events{/privacy}","received_events_url":"https://api.github.com/users/jkbradley/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":12,"created_at":"2015-07-08T18:56:03Z","updated_at":"2015-07-16T06:03:18Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/7294","html_url":"https://github.com/apache/spark/pull/7294","diff_url":"https://github.com/apache/spark/pull/7294.diff","patch_url":"https://github.com/apache/spark/pull/7294.patch"},"body":"This PR copies the RandomForest implementation from spark.mllib to spark.ml.  Note that this includes the DecisionTree implementation, but not the GradientBoostedTrees one (which will come later).\r\n\r\nI essentially copied a minimal amount of code to spark.ml, removed the use of bins (and only used splits), and modified code only as much as necessary to get it to compile.  The spark.ml implementation still uses some spark.mllib classes (privately), which can be moved in future PRs.\r\n\r\nThis refactoring will be helpful in extending the node representation to include more information, such as class probabilities.\r\n\r\nSpecifically:\r\n* Copied code from spark.mllib to spark.ml:\r\n  * mllib.tree.DecisionTree, mllib.tree.RandomForest copied to ml.tree.impl.RandomForest (main implementation)\r\n  * NodeIdCache (needed to use splits instead of bins)\r\n  * TreePoint (use splits instead of bins)\r\n* Added ml.tree.LearningNode used in RandomForest training (needed vars)\r\n* Removed bins from implementation, and only used splits\r\n* Small fix in JavaDecisionTreeRegressorSuite\r\n\r\nCC: @mengxr  @manishamde  @codedeft @chouqin "},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/121842954","html_url":"https://github.com/apache/spark/pull/7294#issuecomment-121842954","issue_url":"https://api.github.com/repos/apache/spark/issues/7294","id":121842954,"user":{"login":"mengxr","id":829644,"avatar_url":"https://avatars.githubusercontent.com/u/829644?v=3","gravatar_id":"","url":"https://api.github.com/users/mengxr","html_url":"https://github.com/mengxr","followers_url":"https://api.github.com/users/mengxr/followers","following_url":"https://api.github.com/users/mengxr/following{/other_user}","gists_url":"https://api.github.com/users/mengxr/gists{/gist_id}","starred_url":"https://api.github.com/users/mengxr/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mengxr/subscriptions","organizations_url":"https://api.github.com/users/mengxr/orgs","repos_url":"https://api.github.com/users/mengxr/repos","events_url":"https://api.github.com/users/mengxr/events{/privacy}","received_events_url":"https://api.github.com/users/mengxr/received_events","type":"User","site_admin":false},"created_at":"2015-07-16T06:03:18Z","updated_at":"2015-07-16T06:03:18Z","body":"@jkbradley I made one pass:\r\n\r\n1. Some of my comments also apply to the old implementation. So I don't think it is necessary to address them in this PR.\r\n2. It is quite hard to track what have been changed, because you merged DecisionTree and RandomForest into one file and removed bins. Next time we should think of a better approach, making the real changes more discoverable.\r\n\r\nExcept some style issues and the random seed, it looks good to me:)"}}