Skip to navigation | Skip to content

Don't believe what you read, says critic


How much of what you read in the journals is flawed? (Image: iStockphoto)

Most published research findings may be false, says a researcher who finds fault with everthing from the way studies are designed to researchers' own prejudices.

Associate Professor John Ioannidis of the University of Ioannina medicine school in Greece, and the US Tufts University medicine school, publishes his analysis in today's issue of the journal PLoS Medicine.

"A research finding is less likely to be true when the studies conducted in the field are smaller; where effect sizes [a measure of how much a risk factor increases a person's risk of disease or how much a treatment is likely to improve a disease] are smaller; when there is a greater number and lesser preselection of tested relationships; and where there is greater flexibility in designs, definitions, outcomes and analytical modes," he writes.

Ioannidis says false results could also result because of prejudices and financial and other interests; or too many scientific teams being involved in the same field "in chase of statistical significance".

He says that simulations show that "for most study designs and settings, it is more likely for a research claim to be false than true".

But in an accompanying editorial, the editors say the publication of "preliminary findings, negative studies, confirmations and refutations" are all an essential part of getting closer to the truth.

Australian expert in evidence-based medicine Professor Don Campbell, director of the Monash Institute of Health Services Research, says while elements of Ioannidis' argument are "plausible", he still thinks his overall argument doesn't make sense.

"Say I gave you a series of connected statements: 'nothing is better than a Rolls Royce', 'my car is better than nothing', 'my car is better than a Rolls Royce'. When you take elements of it on its own they make sense ... But when you look at the joined up argument I don't think it makes sense."

Campbell says it was the same with the study findings.

"There are elements of truth in what he is saying; [however] I question the fundamental assertion that it can be proven that most claimed research findings are false. I don't find evidence to support that in reading this author's paper."

Tags: health, information-technology