{"action":"created","issue":{"url":"https://api.github.com/repos/lme4/lme4/issues/306","labels_url":"https://api.github.com/repos/lme4/lme4/issues/306/labels{/name}","comments_url":"https://api.github.com/repos/lme4/lme4/issues/306/comments","events_url":"https://api.github.com/repos/lme4/lme4/issues/306/events","html_url":"https://github.com/lme4/lme4/issues/306","id":79584969,"number":306,"title":"Description of allow.new.levels does not match implemented behavior.","user":{"login":"tmalsburg","id":3817450,"avatar_url":"https://avatars.githubusercontent.com/u/3817450?v=3","gravatar_id":"","url":"https://api.github.com/users/tmalsburg","html_url":"https://github.com/tmalsburg","followers_url":"https://api.github.com/users/tmalsburg/followers","following_url":"https://api.github.com/users/tmalsburg/following{/other_user}","gists_url":"https://api.github.com/users/tmalsburg/gists{/gist_id}","starred_url":"https://api.github.com/users/tmalsburg/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tmalsburg/subscriptions","organizations_url":"https://api.github.com/users/tmalsburg/orgs","repos_url":"https://api.github.com/users/tmalsburg/repos","events_url":"https://api.github.com/users/tmalsburg/events{/privacy}","received_events_url":"https://api.github.com/users/tmalsburg/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2015-05-22T19:50:02Z","updated_at":"2015-05-28T02:53:12Z","closed_at":null,"body":"The documentation for the parameter `allow.new.levels` in `simulate.merMod` says:\r\n\r\n> ... if TRUE, then the prediction will use the unconditional (population-level) values for data with previously unobserved levels (or NAs).\r\n\r\nHowever, the simulation below shows that this is not true.  In the simulation, I added another 18 subjects to the sleepstudy data set and found that the new subjects have the same statistical properties as old subjects.  Perhaps, someone copied the description from the manpage of `predict.merMod` and forgot to adapt it? \r\n\r\n```R\r\nlibrary(lme4)\r\n\r\n# Fit model using the original data:\r\nd <- sleepstudy\r\nd$Subject <- factor(rep(1:18, each=10))\r\nfm1 <- lmer(Reaction ~ Days + (Days|Subject), d)\r\n\r\n# Add 18 new subjects:\r\nd <- rbind(sleepstudy, sleepstudy)\r\nd$Subject <- factor(rep(1:36, each=10))\r\n\r\n# Simulate old and new subjects 1000 times:\r\nparams <- lapply(1:1000, function(i) {\r\n    message(i)\r\n    d$simulated <- simulate(fm1, newdata=d[-1], allow.new.levels=T)$sim_1\r\n    # A small number of models fail to converge:\r\n    fm.old <- lmer(simulated ~ Days + (Days | Subject), subset(d, Subject%in%1:18))\r\n    fm.new <- lmer(simulated ~ Days + (Days | Subject), subset(d, Subject%in%19:36))\r\n\r\n    c(fixef(fm.old), getME(fm.old, \"theta\"), getME(fm.old, \"sigma\"),\r\n\tfixef(fm.new), getME(fm.new, \"theta\"), getME(fm.new, \"sigma\"))\r\n})\r\n\r\nparams <- do.call(rbind, params)\r\n\r\npar(mfrow=c(1,3), mar=c(2,3,1,1))\r\nplot(density(params[,3]), main=\"Subject intercepts\", ylim=c(0,2))\r\nlines(density(params[,9]), col=\"red\")\r\n\r\nplot(density(params[,5]), main=\"Subject Days\", ylim=c(0,9))\r\nlines(density(params[,11]), col=\"red\")\r\n\r\nplot(density(params[,4]), main=\"Correlation\", ylim=c(0,6))\r\nlines(density(params[,10]), col=\"red\")\r\n\r\nlegend(\"topright\", lty=c(1,1), col=1:2, c(\"old subjects\", \"new subjects\"), bg=\"white\")\r\n```\r\n\r\n**Edit:** Does it actually make sense to distinguish between old and new factor levels for the purpose of simulation?  If not, perhaps it would make sense to remove that parameter.\r\n"},"comment":{"url":"https://api.github.com/repos/lme4/lme4/issues/comments/106154544","html_url":"https://github.com/lme4/lme4/issues/306#issuecomment-106154544","issue_url":"https://api.github.com/repos/lme4/lme4/issues/306","id":106154544,"user":{"login":"tmalsburg","id":3817450,"avatar_url":"https://avatars.githubusercontent.com/u/3817450?v=3","gravatar_id":"","url":"https://api.github.com/users/tmalsburg","html_url":"https://github.com/tmalsburg","followers_url":"https://api.github.com/users/tmalsburg/followers","following_url":"https://api.github.com/users/tmalsburg/following{/other_user}","gists_url":"https://api.github.com/users/tmalsburg/gists{/gist_id}","starred_url":"https://api.github.com/users/tmalsburg/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tmalsburg/subscriptions","organizations_url":"https://api.github.com/users/tmalsburg/orgs","repos_url":"https://api.github.com/users/tmalsburg/repos","events_url":"https://api.github.com/users/tmalsburg/events{/privacy}","received_events_url":"https://api.github.com/users/tmalsburg/received_events","type":"User","site_admin":false},"created_at":"2015-05-28T02:53:12Z","updated_at":"2015-05-28T02:53:12Z","body":"I'd like to add that the implemented default behaviour is actually what I would typically want.  I use `simualate.merMod` for power analyses (following your tutorial on RPubs) and in this context it doesn't make sense to simulate data for the specific individuals that were used for the collection of the pilot data.  The actual experiment will sample from the same subject population but exclude people who participated in the pilot."}}