{"action":"created","issue":{"url":"https://api.github.com/repos/dotnet/roslyn/issues/7356","labels_url":"https://api.github.com/repos/dotnet/roslyn/issues/7356/labels{/name}","comments_url":"https://api.github.com/repos/dotnet/roslyn/issues/7356/comments","events_url":"https://api.github.com/repos/dotnet/roslyn/issues/7356/events","html_url":"https://github.com/dotnet/roslyn/issues/7356","id":121153926,"number":7356,"title":"Test Result Caching and Non Graceful Test Termination","user":{"login":"jeffanders","id":13701677,"avatar_url":"https://avatars.githubusercontent.com/u/13701677?v=3","gravatar_id":"","url":"https://api.github.com/users/jeffanders","html_url":"https://github.com/jeffanders","followers_url":"https://api.github.com/users/jeffanders/followers","following_url":"https://api.github.com/users/jeffanders/following{/other_user}","gists_url":"https://api.github.com/users/jeffanders/gists{/gist_id}","starred_url":"https://api.github.com/users/jeffanders/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jeffanders/subscriptions","organizations_url":"https://api.github.com/users/jeffanders/orgs","repos_url":"https://api.github.com/users/jeffanders/repos","events_url":"https://api.github.com/users/jeffanders/events{/privacy}","received_events_url":"https://api.github.com/users/jeffanders/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/dotnet/roslyn/labels/Area-Compilers","name":"Area-Compilers","color":"5319e7"},{"url":"https://api.github.com/repos/dotnet/roslyn/labels/Bug","name":"Bug","color":"fc2929"}],"state":"open","locked":false,"assignee":{"login":"jaredpar","id":146967,"avatar_url":"https://avatars.githubusercontent.com/u/146967?v=3","gravatar_id":"","url":"https://api.github.com/users/jaredpar","html_url":"https://github.com/jaredpar","followers_url":"https://api.github.com/users/jaredpar/followers","following_url":"https://api.github.com/users/jaredpar/following{/other_user}","gists_url":"https://api.github.com/users/jaredpar/gists{/gist_id}","starred_url":"https://api.github.com/users/jaredpar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jaredpar/subscriptions","organizations_url":"https://api.github.com/users/jaredpar/orgs","repos_url":"https://api.github.com/users/jaredpar/repos","events_url":"https://api.github.com/users/jaredpar/events{/privacy}","received_events_url":"https://api.github.com/users/jaredpar/received_events","type":"User","site_admin":false},"milestone":{"url":"https://api.github.com/repos/dotnet/roslyn/milestones/4","html_url":"https://github.com/dotnet/roslyn/milestones/1.2","labels_url":"https://api.github.com/repos/dotnet/roslyn/milestones/4/labels","id":937402,"number":4,"title":"1.2","description":"This is our 1.2 Milestone.\r\n\r\n<!---\r\n@huboard:{\"order\":2.0}\r\n-->\r\n","creator":{"login":"jaredpar","id":146967,"avatar_url":"https://avatars.githubusercontent.com/u/146967?v=3","gravatar_id":"","url":"https://api.github.com/users/jaredpar","html_url":"https://github.com/jaredpar","followers_url":"https://api.github.com/users/jaredpar/followers","following_url":"https://api.github.com/users/jaredpar/following{/other_user}","gists_url":"https://api.github.com/users/jaredpar/gists{/gist_id}","starred_url":"https://api.github.com/users/jaredpar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jaredpar/subscriptions","organizations_url":"https://api.github.com/users/jaredpar/orgs","repos_url":"https://api.github.com/users/jaredpar/repos","events_url":"https://api.github.com/users/jaredpar/events{/privacy}","received_events_url":"https://api.github.com/users/jaredpar/received_events","type":"User","site_admin":false},"open_issues":582,"closed_issues":223,"state":"open","created_at":"2015-01-15T20:48:55Z","updated_at":"2015-12-09T16:26:51Z","due_on":null,"closed_at":null},"comments":2,"created_at":"2015-12-09T03:53:00Z","updated_at":"2015-12-09T16:27:29Z","closed_at":null,"body":"I recently merged the current master into my fork and ran the tests to ensure the merge had not broken anything.  For reasons unrelated to Roslyn or the tests I decided to terminate the test run.  I am not sure of the correct way to gracefully terminate the tests so I essentially performed a taskkill of the testrunners.  \r\n\r\nNot being aware of the aware of the recent great work by @jaredpar to add caching for test results this left the test cache in a mess and all further test runs were failing horribly in RunTests.exe without any details of what test was failing.  This was because when attempting to print the test failures the `TestResult.ResultsFilePath` was still null which then caused the result printing to fail along with the whole test harness.\r\n\r\nAfter some debugging and then a look through the history for the relevant files I identified that this caching functionality had been added and once I found the cache and cleared it and re-ran the tests everything worked perfectly (and I confirmed that re-running the tests straight away was quick and appeared to use the cache).\r\n\r\nPerhaps this is just a totally inappropriate way to terminate tests and would not be an issue most of the time, but I imagine there may be other situations where tests may be terminated in a non-graceful fashion for various reasons (including, as noted in the code due to a stack overflow exception occurring in a test).\r\n\r\nIt would be good if the caching mechanism could determine if the run was not completed and if so not cache the incomplete results and instead re-run them next time.\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/dotnet/roslyn/issues/comments/163314806","html_url":"https://github.com/dotnet/roslyn/issues/7356#issuecomment-163314806","issue_url":"https://api.github.com/repos/dotnet/roslyn/issues/7356","id":163314806,"user":{"login":"jaredpar","id":146967,"avatar_url":"https://avatars.githubusercontent.com/u/146967?v=3","gravatar_id":"","url":"https://api.github.com/users/jaredpar","html_url":"https://github.com/jaredpar","followers_url":"https://api.github.com/users/jaredpar/followers","following_url":"https://api.github.com/users/jaredpar/following{/other_user}","gists_url":"https://api.github.com/users/jaredpar/gists{/gist_id}","starred_url":"https://api.github.com/users/jaredpar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jaredpar/subscriptions","organizations_url":"https://api.github.com/users/jaredpar/orgs","repos_url":"https://api.github.com/users/jaredpar/repos","events_url":"https://api.github.com/users/jaredpar/events{/privacy}","received_events_url":"https://api.github.com/users/jaredpar/received_events","type":"User","site_admin":false},"created_at":"2015-12-09T16:27:29Z","updated_at":"2015-12-09T16:27:29Z","body":"I agree.  The caching should not take place in the case where xunit exits abnormally.  It should cache failures but not abnormal exits.  Will adjust.  "}}