Replication is the only solution to scientific fraud

If academia is to be cleaned up, the Research Excellence Framework must prize replication over politics and publishing
Dutch scientist Mark Post displays samples of in-vitro meat at the University of Maastricht
Dutch scientist Mark Post displays samples of in-vitro meat at the University of Maastricht Photograph: Francois Lenoir/Reuters

The only way to stop academic malpractice is to eliminate the culture that rewards it. Science today faces a serious challenge in the form of academic fraud, as Alok Jha has highlighted. As easy as it is to decry and punish fraud, the real challenge is to understand what it means, why it happens and how we can prevent it.

At its core, scientific fraud violates every moral and ethical code that scientists stand by. And beyond that, it violates reason itself. Why, after all, would one commit to the lowly salary and long hours of the academy only to sacrifice the one reward that no other career can bestow, that of genuine discovery?

Most practicing scientists, of course, could answer that question in a heartbeat. True discovery is a romance, increasingly dulled by the bureaucracy and corporatisation of university life. Meanwhile, "success" in science massages the ego, leads to promotion, and establishes the winner as a powerful and trusted authority. Added to that, competition for academic posts is peaking and research funding is stretched to the limit.

It comes as no surprise, then, that to survive in academia, let alone thrive, scientists must now game the system in ways that would have appalled our forebears. Outright fraud is just the tip the iceberg. Beneath it churns an ocean of dubious practices that spans the physical, biological and social sciences.

Publish in journals with high impact factors, we're told, even if doing so betrays us as mathematically illiterate. Publish lots (and lots) of papers, we're advised, even if doing so amounts to salami slicing. Oversell our findings to journal editors in the hope of Nature or Science articles that will reap disproportionate professional rewards – rewards that we all admit are ridiculous while in the same breath bestowing them upon each other. And don't bother publishing experiments that show no statistically significant effects because the "best" journals only care about novel positive findings that present a "major advance".

To those outside science these practices may seem egregious, but among scientists they fall into a grey expanse of malpractice that vastly dwarfs outright fraud. Worn down by the pressure to produce rather than discover, many scientists will excuse or engage in various degrees of malpractice. The less scrupulous will push the envelope even further by cherry picking results, massaging analyses, or even baldly fabricating data.

The uncomfortable truth is that the fraudsters aren't that different to the rest of us, they just pushed themselves further along the continuum. If fraud is on the rise it is precisely because we – the scientific community – have nurtured a system that encourages it.

There is much we can do to fix this problem by making science more transparent, for instance by requiring scientists to freely release their raw data. And we should also embrace a culture in which the methodology for scientific experiments is registered publicly prior to the research, as now happens with clinical trials.

To eliminate malpractice entirely, however, we need to get back to basics. The solution, when viewed objectively, is obvious: we need to return to a system in which success depends not only on stacking up piles of "good publications", but on whether our results stand the test of time. Enter our saviour and executioner: replication.

Replication is our best friend because it keeps us honest. In science, false results have a short (albeit potentially damaging) lifespan because regardless of how they come about, other scientists won't be able to reproduce them. On the other hand, true results will be replicated time and again by different scientists. This principle forms the bedrock of all sciences – without it, planes would fall from the sky, medicines wouldn't work, and the Curiosity probe would have landed somewhere near Cocoa Beach, Florida.

Now imagine a science in which research grants and ratings under the Research Excellence Framework were awarded not based on the number of papers a scientist publishes in "top" journals, not on their socio-political connections, and not on their h-index or any other one-dimensional metric of "impact" – but simply on the number of times other scientists could successfully replicate their work.

Imagine a science in which actively replicating the research of other scientists was rewarded with personal success and funding rather than being derided as dull. Imagine a science in which quality and certainty of findings surpassed quantity in every respect, so much so that the amount of individual output was nearly irrelevant. And imagine a science in which "career-making" journals like Nature and Science simply didn't exist – a world in which research was categorised by topic before being peer-reviewed and published in an open forum, freely accessible to all.

Under this system an entire culture of malpractice would be eliminated.

How can we turn this fantasy into reality? The eradication of groupthink must be achieved both from within and, crucially, from without. We need top-down direction from governments and funding agencies to institute the necessary incentives. We need to plan ahead and pressure governments to incorporate measures of genuine research quality into REF 2018. We need to actively pursue markers of replicability, such as those suggested by Brian Nosek. And we need to embrace the open science initiative.

Scientists are among the most trusted professionals in society. But the current wave of fraud reflects a deeper undercurrent of bad practice that threatens to not only undermine public trust in scientists but to stall the engine of science itself. We need a revolution, and we can't fight a rising tide with a few buckets and a brave face – we need to change the orbit of the moon.