In answer to a question about moving between employment in the UK and the US, user StrongBad suggests that "the obsession with the UK REF will likely push your CV in directions that are not ideal for getting a job in the US."

This point struck me as interesting, as in some respects it appears counter intuitive. When I asked why this was the case, he suggested that it would make a good question. I'm now asking that question.

For those unaware of the REF, it is a regularly occurring (and not uncontroversial) nationwide assessment exercise which is used to guide the distribution of funding amongst UK universities.

In order to perform well in the REF, there is a pressure on researchers in the UK to produce a number of publications (four, to be precise) which are deemed to be of a high level. In assessing quality, reviewers are guided to consider whether a piece of work is "world-leading in terms of originality, significance and rigour". Furthermore, since the most recent REF another consideration for assessment has been whether research has "an effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia."

What's therefore not clear to me is why would building a CV containing publications deemed by peers to be world-leading, and conducting research which peers deem to have benefit outside of academia, not be ideal for job prospects in US academia?

  • I hesitated about asking this question as I'm not entirely convinced that I wouldn't flag it myself if I saw it being asked by someone else. Any recommendations about how to make it more SE appropriate are therefore welcome – Ian_Fin Aug 23 '16 at 21:24

The REF credits departments for grant capture and not the individual. This means that large grants with multiple PIs tend to be prioritized over smaller grants to individuals. In terms of the US job market, being able to demonstrate grant capture at the individual level is important in many fields. For example, generally in the US being the only PI on a $200,000 grant is viewed better than being one of ten PIs on a $5,000,000 grant.

The REF "requires" academics to "produce" 4 high "quality" publications over a 4 year period. Only the 4 "best" publications matter. This results in a push for splashy, as opposed to solid, research. The goal for the REF is to make your work look important to people peripherally related to your field and not the people who specialize in your subarea. Authorship across the 4 publications can also be shared so teaming up with a more senior colleague and being a secondary author is very desirable for the REF. For the REF, someone who is a co-author on 4 Nature papers with colleagues at different universities, and nothing else, could easily get a position as an associate professor, if not a full professor. In the US, search committees want to see a body of solid work that is clearly done by the applicant.

Being able to demonstrate impact is also important for the REF. This requires wider engagement with people outside your field. Having research that can be link to a "change in process" is key for the REF. For example, a clinical trial that shows X is better than Y would be considered "high impact" while the development of X would have less impact. Blue skies basic research has essentially no impact and is discuraged. US job searches do not really care about impact.

  • 1
    I don't think that very high quality papers is meant to be "splashy". It means your paper has to be among the top in your field for that year. – Dilworth Aug 24 '16 at 0:15
  • 2
    @Dilworth I think you're understanding what "flashy" means here. Quality isn't a uniform and objective property of a paper. So Strongbad's claim is that there is a consistent difference between what the REF panel and a US hiring commitee will see as quality. I don't have the experience to judge whether that's correct (though it's consistent with my limited knowledge), but it's a plausible claim. – Ben Webster Aug 24 '16 at 0:27
  • 1
    Nice answer. We have a similar assessment system in Italy, which is very criticized for similar reasons. – Massimo Ortolano Aug 24 '16 at 8:19
  • 1
    I have explained in my answer this point: 4* papers are not meant to be papers that are, say, Nature equivalents. But top in their specific areas. Otherwise, in REF 2014, the number of 4* papers would be much less than 20% of papers for many schools. – Dilworth Aug 24 '16 at 16:09
  • 1
    @Dilworth yes, I was on my departments REF 2014 planning committee. Our submissions were internally rated by 3 staff members and externally rated by 2 faculty at comparable universities. There was good agreement both within and across universities. Further, Our predicted numbers of 3* and 4* matched pretty closely to the actual numbers. In general papers in generalist journals (e.g., Nature, PNAS, Royal Society) we 4* and papers in top specialist journals were 3* unless they were really good. – StrongBad Aug 24 '16 at 17:02

I believe you are correct, and that the statement: "the obsession with the UK REF will likely push your CV in directions that are not ideal for getting a job in the US." is mostly incorrect.

On the contrary, the UK REF is likely to have a good impact on the publication habits of UK scholars, as it specifically favors small amount of very high quality (i.e., 4 per four/five years) publications, over high amount of medium or low quality publications.

For instance, to my understanding, for the REF and excellent paper is possibly a technical paper. E.g., a paper that merits publication in the Annals of Mathematics, or Trans. of the AMS in the case of a math department. To my knowledge it is not meant to be papers that are, say, Nature equivalents. Indeed, you can see that many schools according to the 2014 REF metric had produced many top papers (and had it been required to have Nature equivalent publications this would not have been possible).

Note that in the US you will find it hard to get into a good research school without at least some such top papers.

One thing that may hinder a bit basic research is the need to have "impact cases", i.e., research that leads to practical/social/policy-wise impact. But not every academic should have an impact case, and so this may not be a very big problem.

  • 1
    I think getting a US TT job with a publication rate of one publication per year would be nearly impossible in most fields. – StrongBad Aug 24 '16 at 0:00
  • 1
    Having enough very high quality publications is in any case a necessary condition to land a TT position in a good US research university nowadays. So this requirement of REF only helps you. Of course, you are free to have more papers as you wish! – Dilworth Aug 24 '16 at 0:12
  • No one put the panel members really know, but the general consensus is that papers in top specialty journals are 3* unless they have lots of citations and the research is particularly exciting. – StrongBad Aug 24 '16 at 13:37
  • 1
    No department encourages 3* papers. The REF feeds into HEFCE funding which gives 4x the weight to 4* papers as to 3* papers. When you consider the 20% number, you have to realize that papers can be counted multiple times if the authors are at different schools and since authors of 4* publications are generally poached, it wouldn't surprise me if the percentage of 4* items was closer to 10% (maybe even less). Further, the percentages get complicated because people without strong enough records do not get submitted to the REF. – StrongBad Aug 24 '16 at 16:27
  • 1
    Any positive weight to 3* is an encouragement. Of course, 4* is more encouraged. As to the ~10% of 4*, even with your possible explanations of double credits etc., had it been only Nature equivalents that are considered 4*, it would be much less than 10%-20%. – Dilworth Aug 24 '16 at 16:31

Your Answer

