Observational studies in nutrition are a waste of time. Their results should be ignored.
An article in a Lancet sub-journal this month suggested that both very high and very low carbohydrate diets shorten life spans. That may seem odd to casual followers of nutrition science after an article in the Lancet mothership last year said pretty much the opposite – that low carbohydrate (and therefore higher fat, because you have to eat something) diets lengthen life spans. The 2 papers cannot both be right.
The answer, fortunately, is easy. We can safely ignore both studies when choosing our diets because observational studies in nutrition, like these 2, are a waste of time.
In JAMA this week, my Stanford colleague John Ioannidis explains in a brilliant critique of a few more, totally useless observational nutrition studies:
…eating 12 hazelnuts daily (1 oz) would prolong life by 12 years (ie, 1 year per hazelnut), drinking 3 cups of coffee daily would achieve a similar gain of 12 extra years…
I love coffee and hazelnuts, but those conclusions are patently absurd – “implausible” is Dr Ioannidis’ more polite term. The conclusions are absurd because the study methods are absurd.
To see why, take the methods from the pro-carbohydrate study above and replace nutrition with religious variables:
We studied 15,428 adults aged 45-64 years, in four US communities, who completed a prayer questionnaire at enrollment… The primary outcome was all-cause mortality. We investigated the association between the percentage of spiritual energy from Psalm prayers and all-cause mortality… We further examined this association…with data for Psalm prayers reported from seven multinational prospective studies in a meta-analysis. Finally, we assessed whether the substitution of Old or New Testament sources of prayer forPsalm prayers affect mortality.
- Might that study find an association, thanks to a mix of statistical chance and author / publication bias?
- Sure! At a significance threshold of 95%, 1 out of 20 tests any scientist conducts will be flat out wrong.
- Would any sensible scientist believe it?
- I hope not.
- Would that observational conclusion hold up scrutiny in a well designed experiment?
- Only 1 time out of 20 😉
Indeed, we have seen this failure over and over again. For just 2 recent examples, dietary supplements with antioxidants (beta-carotene, vitamin A, vitamin-C etc) and omega-3 fats showed absolutely no health benefit in experimental studies despite “strong” evidence of “benefit” in multiple, published observational studies.
Observational studies in nutrition are a waste of time. Their results should be ignored. As Dr Ioannidis puts it, “Resources for some of these studies could have been better spent on unambiguous, directly manageable threats to health”. Amen. While I harbor no illusions that popular media will grasp at sensational junk no matter its validity, it is high time the scientific community & policymakers got out of the dark ages.
Addendum:
I write all this with the utmost respect for all involved, particularly for Dr Andrew Mente and his colleagues, who, I believe, are diligent, smart, honest scientists. I happen to think their conclusions in the PURE study – that a low carbohydrate, high fat diet is generally healthier than alternatives – are right. However, it would be intellectually dishonest of me to scour just the conclusions with which I don’t agree when the methods are similar. As long as arguments are swayed by observational data, each nutritional-ideological camp has every incentive to keep producing them. I wish all these smart people on all sides found more productive work to do.
