{"action":"created","issue":{"url":"https://api.github.com/repos/pytest-dev/pytest/issues/814","labels_url":"https://api.github.com/repos/pytest-dev/pytest/issues/814/labels{/name}","comments_url":"https://api.github.com/repos/pytest-dev/pytest/issues/814/comments","events_url":"https://api.github.com/repos/pytest-dev/pytest/issues/814/events","html_url":"https://github.com/pytest-dev/pytest/issues/814","id":92789646,"number":814,"title":"Mark for flaky tests","user":{"login":"The-Compiler","id":625793,"avatar_url":"https://avatars.githubusercontent.com/u/625793?v=3","gravatar_id":"","url":"https://api.github.com/users/The-Compiler","html_url":"https://github.com/The-Compiler","followers_url":"https://api.github.com/users/The-Compiler/followers","following_url":"https://api.github.com/users/The-Compiler/following{/other_user}","gists_url":"https://api.github.com/users/The-Compiler/gists{/gist_id}","starred_url":"https://api.github.com/users/The-Compiler/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/The-Compiler/subscriptions","organizations_url":"https://api.github.com/users/The-Compiler/orgs","repos_url":"https://api.github.com/users/The-Compiler/repos","events_url":"https://api.github.com/users/The-Compiler/events{/privacy}","received_events_url":"https://api.github.com/users/The-Compiler/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/pytest-dev/pytest/labels/proposal","name":"proposal","color":"006b75"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":18,"created_at":"2015-07-03T04:40:31Z","updated_at":"2015-08-01T14:24:53Z","closed_at":null,"body":"I'd like to continue the discussion from #776 here. What I said there:\r\n\r\nI agree there should be some difference between \"is expected to fail\" and \"is flaky\".\r\n\r\nI personally use `xfail` for \"I found a bug while writing tests, or found a bug and wrote a test for it - but I don't want to fix it immediately, and I don't want my CI failing because of it\". (I don't have any flaky tests, fortunately)\r\n\r\nIMHO, `xpassed` should be red, and a new `pytest.mark.flaky` marker should be added - then it's also immediately clear for someone reading the tests that this test is *flaky*, not *expected to fail*.\r\n\r\ncc @esiegerman"},"comment":{"url":"https://api.github.com/repos/pytest-dev/pytest/issues/comments/126918376","html_url":"https://github.com/pytest-dev/pytest/issues/814#issuecomment-126918376","issue_url":"https://api.github.com/repos/pytest-dev/pytest/issues/814","id":126918376,"user":{"login":"nicoddemus","id":1085180,"avatar_url":"https://avatars.githubusercontent.com/u/1085180?v=3","gravatar_id":"","url":"https://api.github.com/users/nicoddemus","html_url":"https://github.com/nicoddemus","followers_url":"https://api.github.com/users/nicoddemus/followers","following_url":"https://api.github.com/users/nicoddemus/following{/other_user}","gists_url":"https://api.github.com/users/nicoddemus/gists{/gist_id}","starred_url":"https://api.github.com/users/nicoddemus/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nicoddemus/subscriptions","organizations_url":"https://api.github.com/users/nicoddemus/orgs","repos_url":"https://api.github.com/users/nicoddemus/repos","events_url":"https://api.github.com/users/nicoddemus/events{/privacy}","received_events_url":"https://api.github.com/users/nicoddemus/received_events","type":"User","site_admin":false},"created_at":"2015-08-01T14:24:53Z","updated_at":"2015-08-01T14:24:53Z","body":"Just noticed that `unittest` has a `expectedFailure` decorator with the same semantics proposed here (failures are OK, unexpected passes are not OK). I think this makes a stronger point to a separate marker."}}