{"action":"created","issue":{"url":"https://api.github.com/repos/topepo/caret/issues/318","labels_url":"https://api.github.com/repos/topepo/caret/issues/318/labels{/name}","comments_url":"https://api.github.com/repos/topepo/caret/issues/318/comments","events_url":"https://api.github.com/repos/topepo/caret/issues/318/events","html_url":"https://github.com/topepo/caret/pull/318","id":116241914,"number":318,"title":"Added predleaf to xgbTree for predict","user":{"login":"terrytangyuan","id":4269898,"avatar_url":"https://avatars.githubusercontent.com/u/4269898?v=3","gravatar_id":"","url":"https://api.github.com/users/terrytangyuan","html_url":"https://github.com/terrytangyuan","followers_url":"https://api.github.com/users/terrytangyuan/followers","following_url":"https://api.github.com/users/terrytangyuan/following{/other_user}","gists_url":"https://api.github.com/users/terrytangyuan/gists{/gist_id}","starred_url":"https://api.github.com/users/terrytangyuan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/terrytangyuan/subscriptions","organizations_url":"https://api.github.com/users/terrytangyuan/orgs","repos_url":"https://api.github.com/users/terrytangyuan/repos","events_url":"https://api.github.com/users/terrytangyuan/events{/privacy}","received_events_url":"https://api.github.com/users/terrytangyuan/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-11-11T01:28:06Z","updated_at":"2015-11-11T16:42:44Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/topepo/caret/pulls/318","html_url":"https://github.com/topepo/caret/pull/318","diff_url":"https://github.com/topepo/caret/pull/318.diff","patch_url":"https://github.com/topepo/caret/pull/318.patch"},"body":"Is this the correct way to add it? "},"comment":{"url":"https://api.github.com/repos/topepo/caret/issues/comments/155839061","html_url":"https://github.com/topepo/caret/pull/318#issuecomment-155839061","issue_url":"https://api.github.com/repos/topepo/caret/issues/318","id":155839061,"user":{"login":"topepo","id":5731043,"avatar_url":"https://avatars.githubusercontent.com/u/5731043?v=3","gravatar_id":"","url":"https://api.github.com/users/topepo","html_url":"https://github.com/topepo","followers_url":"https://api.github.com/users/topepo/followers","following_url":"https://api.github.com/users/topepo/following{/other_user}","gists_url":"https://api.github.com/users/topepo/gists{/gist_id}","starred_url":"https://api.github.com/users/topepo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/topepo/subscriptions","organizations_url":"https://api.github.com/users/topepo/orgs","repos_url":"https://api.github.com/users/topepo/repos","events_url":"https://api.github.com/users/topepo/events{/privacy}","received_events_url":"https://api.github.com/users/topepo/received_events","type":"User","site_admin":false},"created_at":"2015-11-11T16:42:43Z","updated_at":"2015-11-11T16:42:43Z","body":"What was the thought behind adding `predleaf` as an argument? I don't know of any way to pass that through from `train`? \r\n\r\nFor the random grid search, you can get rid of the `len` multipliers on `colsample_bytree`, `nrounds`, and `gamma. \r\n\r\nAlso, I did some testing and I don't get the same answers as the code that doesn't use the sub-model trick. Here is my testing code:\r\n\r\n```r\r\nlibrary(caret)\r\nmodelInfo2 <- modelInfo\r\nmodelInfo2$loop <- NULL\r\n\r\n###################################################################\r\n\r\nsmall <- expand.grid(max_depth = c(1, 10),\r\n                     nrounds = c(10, 100, 500),\r\n                     eta = .3,\r\n                     gamma = 0,\r\n                     colsample_bytree = .6,\r\n                     min_child_weight = 1)\r\n\r\n###################################################################\r\n\r\nset.seed(46)\r\ndat <- twoClassSim(200)\r\n\r\nseeds <- vector(mode = \"list\", length = 26)\r\nseeds <- lapply(seeds, function(x) 1:40)\r\n\r\nset.seed(1)\r\nmod1 <- train(Class ~ ., data = dat,\r\n              method = modelInfo,\r\n              tuneGrid = small,\r\n              trControl = trainControl(seeds = seeds, \r\n                                       savePredictions = TRUE,\r\n                                       classProbs = TRUE))\r\n\r\nset.seed(1)\r\nmod2 <- train(Class ~ ., data = dat,\r\n              method = modelInfo2,\r\n              tuneGrid = small,\r\n              trControl = trainControl(seeds = seeds, \r\n                                       savePredictions = TRUE,\r\n                                       classProbs = TRUE))\r\n\r\nall.equal(mod1$results$Accuracy, mod2$results$Accuracy)\r\nsummary(mod1$results$Accuracy-mod2$results$Accuracy)\r\nmod2$times$everything[3]/mod1$times$everything[3]\r\n```\r\nI get:\r\n\r\n```r\r\n> all.equal(mod1$results$Accuracy, mod2$results$Accuracy)\r\n[1] \"Mean relative difference: 0.00694488\"\r\n> summary(mod1$results$Accuracy-mod2$results$Accuracy)\r\n     Min.   1st Qu.    Median      Mean   3rd Qu.      Max. \r\n-0.005201 -0.001801  0.000000  0.001264  0.003199  0.010920 \r\n```\r\n\r\nThey aren't large differences but they shouldn't be there.  \r\n\r\nIt looks like the results are the same for some tuning parameters, specifically when the number of rounds are small. If I re-run `mod1` with the same seed, I get the same results so that isn't the issue. \r\n\r\nI think the problem is in the `xgboost` package itself. For example, I would expect to get the same results doing this:\r\n\r\n```r\r\n> data(agaricus.train, package='xgboost')\r\n> data(agaricus.test, package='xgboost')\r\n> train <- agaricus.train\r\n> test <- agaricus.test\r\n> \r\n> set.seed(1)\r\n> bst1 <- xgboost(data = train$data, label = train$label, max.depth = 2,\r\n+                eta = 1, nthread = 1, nround = 100, objective = \"binary:logistic\")\r\n[0]\ttrain-error:0.046522\r\n[1]\ttrain-error:0.022263\r\n[2]\ttrain-error:0.007063\r\n<snip>\r\n> pred_1_50 <- predict(bst1, test$data, ntreelimit = 50)\r\n> \r\n> set.seed(1)\r\n> bst2 <- xgboost(data = train$data, label = train$label, max.depth = 2,\r\n+                 eta = 1, nthread = 1, nround = 50, objective = \"binary:logistic\")\r\n[0]\ttrain-error:0.046522\r\n[1]\ttrain-error:0.022263\r\n[2]\ttrain-error:0.007063\r\n<snip>\r\n> pred_2_50 <- predict(bst1, test$data)\r\n> \r\n> all.equal(pred_2_50, pred_1_50)\r\n[1] \"Mean relative difference: 0.0001912147\"\r\n```\r\n\r\nI'll submit a ticket to xgboost on this. \r\n"}}