Lies, Damn Lies and Statistics

Mark Twain and others have said:  “There are Lies, there are Damn Lies, and then there are Statistics”.  That is the danger when we use Statistics (Meta-Analyses) as a basis for “Evidence-Based” Medicine.  A recent meta-analysis looking at Vitamin D has reached a gaggle of questionable conclusions.  The news media have swallowed it hook, line and sinker, and want you to do so as well.

This meta-analysis re-analyzed 40 old studies which were not designed to track actual 25-hydroxy Vitamin D levels, and used doses of Vitamin D that we now know are too low to be effective.  They looked for diseases which were not specifically tracked, but might be mentioned as incidental findings.  None of these studies individually showed a “statistically significant” finding.  But if you combine enough different studies, the numbers become large enough that “statistical significance” can happen.

What is statistical significance?  A 95% chance that what you found is correct – provided the information you used is accurate (a big problem in most meta-analyses).  Therefore, there is at least a 5% chance that the finding is flat out wrong!  Bummer.

Current ethical standards make it very difficult to organize large, randomized, prospective, placebo controlled, multiyear studies which place half of the participants randomly on a non-treatment which the researchers believe will lead to disease or disability.  Ethical standards are good.

But that leaves us with meta-analyses.  Unfortunately, all they do is crunch the numbers again on old studies, hoping that mixing apples, oranges, lemons, and limes will prove something useful.  Questionable medical data.

I prefer “observational” studies which can prove an association, although not necessarily cause and effect.  See my recent post about Low Vitamin D Levels and Cancer.  Also Vitamin D Prevents Fractures – Only If You Take Enough.

Jay Ginther, MD

Categorised in: Nutrition