{"action":"opened","issue":{"url":"https://api.github.com/repos/hadley/testthat/issues/267","labels_url":"https://api.github.com/repos/hadley/testthat/issues/267/labels{/name}","comments_url":"https://api.github.com/repos/hadley/testthat/issues/267/comments","events_url":"https://api.github.com/repos/hadley/testthat/issues/267/events","html_url":"https://github.com/hadley/testthat/issues/267","id":86622027,"number":267,"title":"skip_on_cran() causing test failures when NOT_CRAN='false'","user":{"login":"zachmayer","id":581590,"avatar_url":"https://avatars.githubusercontent.com/u/581590?v=3","gravatar_id":"","url":"https://api.github.com/users/zachmayer","html_url":"https://github.com/zachmayer","followers_url":"https://api.github.com/users/zachmayer/followers","following_url":"https://api.github.com/users/zachmayer/following{/other_user}","gists_url":"https://api.github.com/users/zachmayer/gists{/gist_id}","starred_url":"https://api.github.com/users/zachmayer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zachmayer/subscriptions","organizations_url":"https://api.github.com/users/zachmayer/orgs","repos_url":"https://api.github.com/users/zachmayer/repos","events_url":"https://api.github.com/users/zachmayer/events{/privacy}","received_events_url":"https://api.github.com/users/zachmayer/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2015-06-09T14:40:05Z","updated_at":"2015-06-09T14:40:05Z","closed_at":null,"body":"The skip functions are supposed to silently skip tests, rather than raising an error and cause test failures.  I'm working to isolate the exact cause of the error, as it's only caused by `skip_on_cran()` and isn't triggered by `skip`.  Furthermore, it only shows up when `devtools::check()` is run, but not when I manually run the tests.\r\n\r\nFor example, this code runs fine:\r\n```{R}\r\nSys.setenv(NOT_CRAN='false')\r\ncontext(\"Test skips are working correctly\")\r\ntest_that(\"Skips work correctly\", {\r\n  skip(\"Basic skip failed\")\r\n  })\r\ntest_that(\"Skips work correctly\", {\r\n  skip_on_travis()\r\n  })\r\ntest_that(\"Skips work correctly\", {\r\n  skip_on_cran()\r\n  })\r\n```\r\n\r\nBut then `devtools::check()` eventually yields:\r\n```{R}\r\n* checking tests ...\r\n  Running ‘testthat.R’ [16s/14s]\r\n ERROR\r\nRunning the tests in ‘tests/testthat.R’ failed.\r\nLast 13 lines of output:\r\n  Loading required package: survival\r\n  \r\n  Attaching package: 'survival'\r\n  \r\n  The following object is masked from 'package:caret':\r\n  \r\n      cluster\r\n  \r\n  Loading required package: splines\r\n  Loading required package: parallel\r\n  Loaded gbm 2.1.1\r\n  Error: On CRAN\r\n  Execution halted\r\n```\r\n Here's my session info.  I'll try to pare down the list of packages to the minimal set required to cause the error.  It's entirely possible something weird is happening inside `caretEnsemble` too:\r\n\r\n```{R}\r\n> sessionInfo()\r\nR version 3.2.0 (2015-04-16)\r\nPlatform: x86_64-apple-darwin13.4.0 (64-bit)\r\nRunning under: OS X 10.9.5 (Mavericks)\r\n\r\nlocale:\r\n[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8\r\n\r\nattached base packages:\r\n[1] parallel  splines   stats     graphics  grDevices utils     datasets  methods   base     \r\n\r\nother attached packages:\r\n [1] caretEnsemble_1.0.1  klaR_0.6-12          lintr_0.3.0.9002     MASS_7.3-40          plyr_1.8.2           ipred_0.9-4          nnet_7.3-9          \r\n [8] kernlab_0.9-20       gbm_2.1.1            survival_2.38-1      rpart_4.1-9          randomForest_4.6-10  caret_6.0-47         ggplot2_1.0.1       \r\n[15] lattice_0.20-31      testthat_0.10.0.9000\r\n\r\nloaded via a namespace (and not attached):\r\n [1] gtools_3.5.0        reshape2_1.4.1      pbapply_1.1-1       colorspace_1.2-6    mgcv_1.8-6          prodlim_1.5.1       e1071_1.6-4         nloptr_1.0.4       \r\n [9] foreach_1.4.2       lava_1.4.0          stringr_1.0.0       combinat_0.0-8      munsell_0.4.2       gtable_0.1.2        devtools_1.8.0.9000 caTools_1.17.1     \r\n[17] codetools_0.2-11    memoise_0.2.1       knitr_1.10.5        labeling_0.3        SparseM_1.6         rex_1.0.1           quantreg_5.11       pbkrtest_0.4-2     \r\n[25] class_7.3-12        proto_0.3-10        Rcpp_0.11.6         scales_0.2.4        BradleyTerry2_1.0-6 lme4_1.1-7          gridExtra_0.9.1     digest_0.6.8       \r\n[33] stringi_0.4-1       brglm_0.5-9         grid_3.2.0          tools_3.2.0         bitops_1.0-6        magrittr_1.5        lazyeval_0.1.10     crayon_1.3.0       \r\n[41] car_2.0-25          Matrix_1.2-1        pROC_1.8            minqa_1.2.4         roxygen2_4.1.1      iterators_1.0.7     igraph_0.7.1        nlme_3.1-120       \r\n[49] compiler_3.2.0 \r\n```"}}