Great article with a silly title: Lies, Damned Lies, and Medical Science.

Quote from the hero of the piece: something to help us recast scientific research for the future:

“Science is a noble endeavor, but it’s also a low-yield endeavor,” he says. “I’m not sure that more than a very small percentage of medical research is ever likely to lead to major improvements in clinical outcomes and quality of life. We should be very comfortable with that fact.”

Ioannidis put together a model

“that predicted, in different fields of medical research, rates of wrongness roughly corresponding to the observed rates at which findings were later convincingly refuted: 80 percent of non-randomized studies (by far the most common type) turn out to be wrong, as do 25 percent of supposedly gold-standard randomized trials, and as much as 10 percent of the platinum-standard large randomized trials.”

“When he tested his model on 49 articles {which have led to significant changes in medical practice}, 45 claimed to have uncovered effective interventions. Thirty-four of these claims had been retested, and 14 of these, or 41 percent, had been convincingly shown to be wrong or significantly exaggerated”

The 41% is only of the retested claims, so we could be inclined to give the others the benefit of the doubt. The positive note here is that serious long-term studies appear to be pretty good at getting to the truth (10% error) vs non-controlled trials (20% truth).

There may be some language here that can help change the perception of science as putting up an absolute truth to battle with non-science.