Great article with a silly title: Lies, Damned Lies, and Medical Science.
Quote from the hero of the piece: something to help us recast scientific research for the future:
“Science is a noble endeavor, but it’s also a low-yield endeavor,†he says. “I’m not sure that more than a very small percentage of medical research is ever likely to lead to major improvements in clinical outcomes and quality of life. We should be very comfortable with that fact.â€
Ioannidis put together a model
“that predicted, in different fields of medical research, rates of wrongness roughly corresponding to the observed rates at which findings were later convincingly refuted: 80 percent of non-randomized studies (by far the most common type) turn out to be wrong, as do 25 percent of supposedly gold-standard randomized trials, and as much as 10 percent of the platinum-standard large randomized trials.”
“When he tested his model on 49 articles {which have led to significant changes in medical practice}, 45 claimed to have uncovered effective interventions. Thirty-four of these claims had been retested, and 14 of these, or 41 percent, had been convincingly shown to be wrong or significantly exaggerated”
The 41% is only of the retested claims, so we could be inclined to give the others the benefit of the doubt. The positive note here is that serious long-term studies appear to be pretty good at getting to the truth (10% error) vs non-controlled trials (20% truth).
There may be some language here that can help change the perception of science as putting up an absolute truth to battle with non-science.