{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/4819","labels_url":"https://api.github.com/repos/apache/spark/issues/4819/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/4819/comments","events_url":"https://api.github.com/repos/apache/spark/issues/4819/events","html_url":"https://github.com/apache/spark/pull/4819","id":59294852,"number":4819,"title":"[SPARK-6025] [MLlib] Add helper method to efficiently compute error in GBT's","user":{"login":"MechCoder","id":1867024,"avatar_url":"https://avatars.githubusercontent.com/u/1867024?v=3","gravatar_id":"","url":"https://api.github.com/users/MechCoder","html_url":"https://github.com/MechCoder","followers_url":"https://api.github.com/users/MechCoder/followers","following_url":"https://api.github.com/users/MechCoder/following{/other_user}","gists_url":"https://api.github.com/users/MechCoder/gists{/gist_id}","starred_url":"https://api.github.com/users/MechCoder/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/MechCoder/subscriptions","organizations_url":"https://api.github.com/users/MechCoder/orgs","repos_url":"https://api.github.com/users/MechCoder/repos","events_url":"https://api.github.com/users/MechCoder/events{/privacy}","received_events_url":"https://api.github.com/users/MechCoder/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":11,"created_at":"2015-02-27T21:11:30Z","updated_at":"2015-03-02T19:56:47Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/4819","html_url":"https://github.com/apache/spark/pull/4819","diff_url":"https://github.com/apache/spark/pull/4819.diff","patch_url":"https://github.com/apache/spark/pull/4819.patch"},"body":"While computing the error, with and without validation, for every iteration, the feature prediction of the previous trees was not being cached, which lead to re computation."},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/76799858","html_url":"https://github.com/apache/spark/pull/4819#issuecomment-76799858","issue_url":"https://api.github.com/repos/apache/spark/issues/4819","id":76799858,"user":{"login":"MechCoder","id":1867024,"avatar_url":"https://avatars.githubusercontent.com/u/1867024?v=3","gravatar_id":"","url":"https://api.github.com/users/MechCoder","html_url":"https://github.com/MechCoder","followers_url":"https://api.github.com/users/MechCoder/followers","following_url":"https://api.github.com/users/MechCoder/following{/other_user}","gists_url":"https://api.github.com/users/MechCoder/gists{/gist_id}","starred_url":"https://api.github.com/users/MechCoder/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/MechCoder/subscriptions","organizations_url":"https://api.github.com/users/MechCoder/orgs","repos_url":"https://api.github.com/users/MechCoder/repos","events_url":"https://api.github.com/users/MechCoder/events{/privacy}","received_events_url":"https://api.github.com/users/MechCoder/received_events","type":"User","site_admin":false},"created_at":"2015-03-02T19:56:47Z","updated_at":"2015-03-02T19:56:47Z","body":"@jkbradley I completely understand the logic, but I still do not get the design aspect. (Sorry for being slow)\r\nWhat should evaluateEachIteration return? The error corresponding to every iteration right?\r\n\r\n> That may require adding some internal API to ensembles to permit prediction from a pre-computed sum of trees' predictions.\r\nThat is exactly what this PR does.\r\n(computeInitialResidual) https://github.com/apache/spark/pull/4819/files#diff-7b5c1db0b1926a36f418b53fcf807db0R160 This is the initial cache.\r\n(updateResidual) https://github.com/apache/spark/pull/4819/files#diff-7b5c1db0b1926a36f418b53fcf807db0R170 and this adds up the prediction for every iteration.\r\n\r\nNote that doing this prevents the subtraction of the true label from the predicted value for every iteration.\r\n"}}