{"action":"created","issue":{"url":"https://api.github.com/repos/junit-team/junit/issues/1130","labels_url":"https://api.github.com/repos/junit-team/junit/issues/1130/labels{/name}","comments_url":"https://api.github.com/repos/junit-team/junit/issues/1130/comments","events_url":"https://api.github.com/repos/junit-team/junit/issues/1130/events","html_url":"https://github.com/junit-team/junit/pull/1130","id":70943605,"number":1130,"title":"Add Ordering, Orderable and @OrderWith","user":{"login":"kcooney","id":478190,"avatar_url":"https://avatars.githubusercontent.com/u/478190?v=3","gravatar_id":"","url":"https://api.github.com/users/kcooney","html_url":"https://github.com/kcooney","followers_url":"https://api.github.com/users/kcooney/followers","following_url":"https://api.github.com/users/kcooney/following{/other_user}","gists_url":"https://api.github.com/users/kcooney/gists{/gist_id}","starred_url":"https://api.github.com/users/kcooney/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kcooney/subscriptions","organizations_url":"https://api.github.com/users/kcooney/orgs","repos_url":"https://api.github.com/users/kcooney/repos","events_url":"https://api.github.com/users/kcooney/events{/privacy}","received_events_url":"https://api.github.com/users/kcooney/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":31,"created_at":"2015-04-25T17:46:40Z","updated_at":"2015-05-11T18:50:53Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/junit-team/junit/pulls/1130","html_url":"https://github.com/junit-team/junit/pull/1130","diff_url":"https://github.com/junit-team/junit/pull/1130.diff","patch_url":"https://github.com/junit-team/junit/pull/1130.patch"},"body":"These APIs allow arbitrary ordering of tests, including randomization."},"comment":{"url":"https://api.github.com/repos/junit-team/junit/issues/comments/101015315","html_url":"https://github.com/junit-team/junit/pull/1130#issuecomment-101015315","issue_url":"https://api.github.com/repos/junit-team/junit/issues/1130","id":101015315,"user":{"login":"dsaff","id":46155,"avatar_url":"https://avatars.githubusercontent.com/u/46155?v=3","gravatar_id":"","url":"https://api.github.com/users/dsaff","html_url":"https://github.com/dsaff","followers_url":"https://api.github.com/users/dsaff/followers","following_url":"https://api.github.com/users/dsaff/following{/other_user}","gists_url":"https://api.github.com/users/dsaff/gists{/gist_id}","starred_url":"https://api.github.com/users/dsaff/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dsaff/subscriptions","organizations_url":"https://api.github.com/users/dsaff/orgs","repos_url":"https://api.github.com/users/dsaff/repos","events_url":"https://api.github.com/users/dsaff/events{/privacy}","received_events_url":"https://api.github.com/users/dsaff/received_events","type":"User","site_admin":false},"created_at":"2015-05-11T18:50:53Z","updated_at":"2015-05-11T18:50:53Z","body":"I'm worried I'm going to miss some points, but here goes:\r\n\r\n1) I'd like to come back to the ordering question in a minute, since I think the problem is different depending on what the base language for modifiers is.\r\n\r\n2) I agree that configuring one annotation with another is ugly and hard to figure out.  That's one reason why we went with annotated fields for Rules.  \r\n\r\nHere's my pitch for Modifier fields, as at least one of the ways they can be specified.  (a) The drawbacks don't seem that bad to me (b) There are clear advantages.  (c) I think there are some clear disadvantages to meta-annotations.\r\n\r\na) The drawbacks don't seem that bad to me.\r\n\r\nI think that annotated fields have worked for Rules.  I know that _some_ people would like a shorthand (and there's nothing preventing us from adding one, for when the shorthand works), but in general, I would say that I've been pretty happy with that decision, and I still like that as a solution here.  So then your RunFailingTests example becomes:\r\n\r\n@ModifyWith public static Modifier failuresFirst = new RunFailuresFirst(\"/tmp/runHistory\");\r\n\r\nYes, there's \"public static Modifier [name]\", none of which add much to the meaning, on one hand.  But Java is a boilerplate language; the more we try to find tricks to elide boilerplate, the less the result is going with the grain of Java, IMHO.\r\n\r\nb) There are clear advantages of annotated fields:\r\n\r\nFields can be reused:\r\n\r\n@ModifyWith public static Modifier OUR_TEAM_MODIFIERS = TeamConfig.OUR_TEAM_MODIFIERS.\r\n\r\nAnd they can include code:\r\n\r\n@ModifyWith public static Modifier sortByNumberOfCharactersInName = new Sorter(new Comparator() {\r\n  public int compare(Description a, Description b) {\r\n    return a.getName().length() - b.getName().length();\r\n  }\r\n}\r\n\r\nThey can provide obvious callback points for related code:\r\n\r\n@ModifyWith public static RunFailuresFirst failuresFirst = new RunFailuresFirst(\"/tmp/runHistory\")\r\n\r\n... later ...\r\n\r\n// This test is super flaky; I don't want it always popping to the top of the test queue\r\n// TODO(saff, 1/4/2002: fix this flaky test)\r\nfailuresFirst.dontPrioritizeThisTestIfItFails();\r\n\r\nc) There are clear disadvantages of meta-annotations, _at this level_\r\n\r\nI have sometimes been in the situation in which someone used a test framework in an extremely clever way that I couldn't find without serious digging.  Because of this cleverness, tests failed in ways that looked like they were finding bugs in the production code, or (much worse!) tests passed that should have failed, because it wasn't clear that there was a convention that needed to be followed for the tests to have any meaning at all.\r\n\r\nIn most cases, this test-case cleverness was implemented by a software engineer that had long since left the team (sometimes for reasons not entirely unconnected with their tendency to produce over-clever solutions (yep, you guessed it, they were promoted)).\r\n\r\nBecause of this, I really really really don't want to be in a position where to truly understand what a test is doing, I need to start _also_ looking at every annotation on every annotation of every superclass of my test case, as well as any @RunWiths on those classes; adding this as a vector for invisible cleverness is, in my opinion, very much a regression.\r\n\r\nNow, note that my primary concern here is that this is a _top level_ API.  Deep in the Theories API, for example, there are meta-annotations for selecting parameters, which look very similar indeed to the meta-annotations you're suggesting for Parameterized.  I think that's a different situation, because it is part of how one particular optional runner works, not all runners, and I'd be much more open to such a change there.\r\n\r\nNEW 3) The point of passing the Class was for finding things through reflection that weren't in the original description.  But, similar to (1), I think I'd rather dig in on the question of simple class-level annotations vs. class-level meta-annotations vs annotated fields before going to far on this question.\r\n\r\n"}}