If it's too good to be true...

Have you heard that eating chocolate can accelerate your weight-loss program? I have. The finding has been widely reported in all sorts of well-known publications, ranging from Prevention and Shape magazines to Huffington Post and Daily Mail Australia. This was discovered by a team of German researchers who conducted a study and found that those who followed a low-carb diet including a small daily chocolate bar lost weight faster than those following a low-carb diet alone.

The only problem?

It was all a setup.

The lead researcher, Dr. Johannes Bohannon, is actually Dr. John Bohannon. He's a journalist, and his Ph.D. is in the molecular biology of bacteria, not human nutrition. He'll tell you more about how and why he pulled off this huge "gotcha" here: http://io9.com/i-fooled-millions-into-thinking-chocolate-helps-weight-1707251800.

NPR does a good rundown of the whole story and some of the lessons we can take away from it, but let's look at a few factors that even a non-scientist like me could catch if we had looked into the claims a little bit:

• How many people participated in the study? Only 15 people actually completed this study. As Bohannon points out in his "confession," most scientists will look with suspicion on any study without at least 30 subjects. 

• How long was the study? This one only lasted three weeks. Has your weight fluctuated a bit in the last three weeks? Mine has. Things like how much you've been sweating recently, how much water you've been drinking, and what point you're at in your menstrual cycle can all affect weight in the short-run.

• Did the researchers control for age, activity level, gender, or, well, anything? It doesn't seem so.

So, right away, this is a poorly-designed study. Also, there's the small matter of the lead researcher, Dr. Johannes Bohannon, not seeming to have any Google presence at all prior to the publication of this study. That's odd.

So, without knowing very much science at all, and without having even heard of "p-hacking," we could have easily found enough that we COULD understand to be skeptical of these claims.

The problem is that so many busy reporters latched onto the press release without bringing any critical thinking to bear on the study itself. This is one of the reasons that we stress the importance of evaluating sources when we talk about information and research skills. Even a publication with "journal" or "archives" in the title isn't necessarily reliable. And certainly, it seems, we can't rely on frazzled journalists to do our evaluation for us.

Now I want a Hershey bar...

Public domain images from Pixabay.com