{"push_id":717845768,"size":1,"distinct_size":0,"ref":"refs/heads/release","head":"3f2503c490e189a394457fa00fff01d0d6307793","before":"ef7c2ce3ccd8098faf5764a6bf25adb375da1aa7","commits":[{"sha":"3f2503c490e189a394457fa00fff01d0d6307793","author":{"email":"474ba67bdb289c6263b36dfd8a7bed6c85b04943@floehopper.org","name":"James Mead"},"message":"Run setup for every regression test if teardown hooks installed\n\nWhen the regression tests are run in the same process as the main test suite,\n`test/test_helper.rb` installs a couple of teardown hooks for all subclasses of\n`Minitest::Test` which are not installed when the regression tests are run in a\nseparate process. The two teardown hooks are as follows:\n\n* WebMock teardown hook installed by `require 'webmock/minitest'` which calls\n`WebMock.reset!`\n* Custom teardown hook calls `Timecop.return`\n\nWith these teardown hooks installed, the optimisation [1] of only running the\n`setup` block once was no longer safe, because everything that was setup before\nthe first test was torn down at the end of the first test and never setup again.\nAnd thus we saw build failure like this one [2] where the expected Content API\nstubs were not in place for subsequent tests.\n\nTypically this didn't happen very often, because developers have tended to run\nthe regression tests separately on their local machine and, as long as the\nrelevant file checksums had been updated, the main CI build didn't run any\nregression tests. However, clearly it was not a good state of affairs.\n\nUnfortunately, because both of these teardown hooks are installed using\n`alias_method_chain` and the WebMock one is within library code, it would've\nbeen awkward to disable them just for the duration of the regression tests and\nre-instate them afterwards.\n\nInstead, I've chosen to detect when they are installed and in this case disable\nthe optimisation of only running the `setup` block once in the regression\ntests. Unfortunately this will make the main test suite take a bit longer if\nany regression tests need to be run. However, since this is a less common\nscenario I think this is good enough for now.\n\n[1]: c86ba038821012a70cb5d8ee3c462fd6f9c5e53c\n[2]: https://ci-new.alphagov.co.uk/job/govuk_smartanswers_branches/2424/consoleFull","distinct":false,"url":"https://api.github.com/repos/alphagov/smart-answers/commits/3f2503c490e189a394457fa00fff01d0d6307793"}]}