{"action":"created","issue":{"url":"https://api.github.com/repos/luisobo/Nocilla/issues/105","labels_url":"https://api.github.com/repos/luisobo/Nocilla/issues/105/labels{/name}","comments_url":"https://api.github.com/repos/luisobo/Nocilla/issues/105/comments","events_url":"https://api.github.com/repos/luisobo/Nocilla/issues/105/events","html_url":"https://github.com/luisobo/Nocilla/issues/105","id":54643690,"number":105,"title":"Sporadic results (XCTest) when running simultaneous tests","user":{"login":"bredfield","id":490270,"avatar_url":"https://avatars.githubusercontent.com/u/490270?v=3","gravatar_id":"","url":"https://api.github.com/users/bredfield","html_url":"https://github.com/bredfield","followers_url":"https://api.github.com/users/bredfield/followers","following_url":"https://api.github.com/users/bredfield/following{/other_user}","gists_url":"https://api.github.com/users/bredfield/gists{/gist_id}","starred_url":"https://api.github.com/users/bredfield/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bredfield/subscriptions","organizations_url":"https://api.github.com/users/bredfield/orgs","repos_url":"https://api.github.com/users/bredfield/repos","events_url":"https://api.github.com/users/bredfield/events{/privacy}","received_events_url":"https://api.github.com/users/bredfield/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2015-01-17T00:51:02Z","updated_at":"2015-07-15T13:32:51Z","closed_at":null,"body":"Using Xcode 6.1, Swift, XCTest, I've got Nocilla working nicely when tests are run individually. As soon as I try to run multiple tests that use Nocilla stubs, they tests that previously succeeded on their own won't work. \r\n\r\nFor example, the following test works well on its own:\r\n```\r\nfunc testUserLoginFailure404() {\r\n        let expectation = expectationWithDescription(\"Login Failure\")\r\n        let testEmail = \"loginfailure@email.com\"\r\n        let testPassword = \"loginFailure\"\r\n        let testToken = \"loginFailureToken\"\r\n        let loginURLString = BRServer.baseURLString + BRServer.Login([\"email\":testEmail, \"password\": testPassword]).fullURL.path! + \"/\"\r\n        \r\n        stubRequest(\"POST\", loginURLString)\r\n            .withBody(\"password=\\(testPassword)&grant_type=password&username=\\(testEmail)\")\r\n            .andReturn(404)\r\n            .withBody(\"{}\")\r\n        \r\n        BRSession.createSessionFromLogin((email: testEmail, password:testPassword), success: { (token) -> Void in\r\n            XCTFail(\"Login should not have succeeded\")\r\n            expectation.fulfill()\r\n        }) { (statusCode, response) -> Void in\r\n                XCTAssert(statusCode == 404, \"should 404\")\r\n                expectation.fulfill()\r\n        }\r\n        \r\n        waitForExpectationsWithTimeout(10.0, handler: nil)\r\n}\r\n```\r\n\r\nHowever, when run alongside other session tests in the same format, the expectation will sporadically never be met (success/fail blocks aren't executed).\r\n\r\nI've made sure that I'm using a separate request URL for each test, so that stubs don't interfere with each other in that way. Any ideas as to why it's not working?\r\n\r\nNote: Here's what I'm doing in `setUp` and `tearDown`:\r\n```\r\n    override func setUp() {\r\n        super.setUp()\r\n\r\n        LSNocilla.sharedInstance().start()\r\n    }\r\n    \r\n    override func tearDown() {\r\n        LSNocilla.sharedInstance().clearStubs()\r\n        super.tearDown()\r\n    }\r\n``` \r\nThanks!"},"comment":{"url":"https://api.github.com/repos/luisobo/Nocilla/issues/comments/121618267","html_url":"https://github.com/luisobo/Nocilla/issues/105#issuecomment-121618267","issue_url":"https://api.github.com/repos/luisobo/Nocilla/issues/105","id":121618267,"user":{"login":"paudav","id":2089039,"avatar_url":"https://avatars.githubusercontent.com/u/2089039?v=3","gravatar_id":"","url":"https://api.github.com/users/paudav","html_url":"https://github.com/paudav","followers_url":"https://api.github.com/users/paudav/followers","following_url":"https://api.github.com/users/paudav/following{/other_user}","gists_url":"https://api.github.com/users/paudav/gists{/gist_id}","starred_url":"https://api.github.com/users/paudav/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/paudav/subscriptions","organizations_url":"https://api.github.com/users/paudav/orgs","repos_url":"https://api.github.com/users/paudav/repos","events_url":"https://api.github.com/users/paudav/events{/privacy}","received_events_url":"https://api.github.com/users/paudav/received_events","type":"User","site_admin":false},"created_at":"2015-07-15T13:32:50Z","updated_at":"2015-07-15T13:32:50Z","body":"Hi @bredfield,\r\n\r\nTL;DR:  This is probably caused by the exception thrown [here](https://github.com/luisobo/Nocilla/blob/master/Nocilla/LSNocilla.m#L79).  When this gets thrown, subsequent requests in the same test run to correctly stubbed URLs seem to fail.\r\n\r\nI'm working in Objective-C, but I've run into a similar issue.  Since you've changed test suites, this probably isn't a big deal for you, but I'll leave this post here for anybody else who is seeing inconsistent results during test runs.\r\n\r\nIn my case, our class under test is using NSURLConnection with delegate callbacks directed to an NSOperationQueue.\r\n\r\nAs described in issue #103, the exception thrown at https://github.com/luisobo/Nocilla/blob/master/Nocilla/LSNocilla.m#L79 basically wrecks all of the tests that run after the exception is thrown.\r\n\r\nIn Xcode, the tests seem to run in alphabetical order of the test method names--Depending on the order of tests your XCTest class implementation, this could make things appear to be \"randomly\" failing.  (For fun, reorder the test methods in your file alphabetically, and see if a more clear failure pattern appears).\r\n\r\nIn our case, the test that caused the exception was intentionally hitting a non-stubbed URL in order to test behavior with non-responsive requests.  By design, this test succeeded after checking some queuing behaviors related to a bunch of open requests (to non-stubbed URLs).  Inside Nocilla, of course, this throws an exception that we never see, and subsequent tests with correctly stubbed requests fail.\r\n\r\nThe result, of course, is that if you run each of the tests individually, they pass.  If you run them all together, the non-stubbed request in the middle of the test run breaks every subsequent test.\r\n\r\nYou can detect this condition by setting an [exception breakpoint](http://stackoverflow.com/questions/17802662/exception-breakpoint-in-xcode).  If you try this and it breaks into LSNocilla.m during the test run, you've sent a request to a non-stubbed URL.\r\n\r\nGood Luck,\r\nPaul B. Davis"}}