The Threads of Bad Science

How do you extract bad science from your head?

During one of my previous jobs, I had to pore over the research on eating behavior.

It didn't take me long to stumble over Brian Wansink's prolific oeuvre. In classic behavioral economist fashion, he published many quirky experiments illuminating subtle factors that influence how we eat. Among his intuitively appealing findings:

• Big plates (and bowls) lead to over-eating – and the size of our plates and bowls have steadily increased over time.
• Small packages lead people to consume smaller fewer calories.
• Distractions lead to over-eating.
• Social signals, like whether your dining companion orders another drink or a fancy dessert, influence what we eat.

His theoretical account of many of these effects came down to two factors: norms and monitoring.

Norms establish what is expected – normal – to eat in a given setting. The size of a package can establish a standard serving size. The meals a restaurant prepares establish what a person is expected to eat. The people who eat around you illustrate what is socially acceptable to eat.

Monitoring is about the opportunities you have to pause, notice your eating, and, if needed, stop. Distracted eating reduces monitoring (and, therefore, increases consumption). You're not paying attention to signals that you should stop eating (e.g., feeling full, the food not tasting as good anymore). Eating chips from a large communal bowl leads to fewer monitoring events (increasing consumption), while eating chips from smaller, individual bowls leads to more monitoring events (decreasing consumption) because finishing the small bowl leads to a decision about whether to get more food.

He wasn't the only researcher exploring how subtle forces influence eating behaviors, of course. But he was a leader in interpreting eating behavior under a "predictably irrational" framework. And he argued for subtle behavioral interventions to help solve the obesity crisis in the States: smaller packages, smaller dishware, fewer stored unhealthy snacks, more visible healthy snacks, vegetables served "family style" (leading to less monitoring), desserts and snacks in personal dishes (leading to more monitoring), etc.

There was just one problem: he was juking the stats. Several of his papers came under scrutiny in recent years, revealing a pattern of sloppy data handling and even deliberate data manipulation leading to results that simply could not be true. Although Wansink admitted no wrong-doing, he retired from Cornell after the controversy.

Part of me believes that Wansink is correct about norms and monitoring – even though I know that some of the evidence supporting his ideas have been undercut. It makes too much "cognitive psychology" sense. Which is a dangerous thing.

When someone has done sloppy or fraudulent research, it's not simply a matter of retracting their questionable articles and moving on. Their findings – not to mention their methods – have already influenced how people think about the phenomenon. In the same way that jurors can't un-see evidence that is later ruled inadmissible or irrelevant, it's hard for me to interpret eating behavior without hearing a little bit of Wansink in my head.

Without picking through each of Wansink's articles, it's hard to judge what should fall and what should stand. The man authored hundreds of articles and his articles have been cited tens of thousands of times, weaving the thread of his work throughout the literature on eating behavior. Only about 20 or so papers have been retracted, but that's more than enough to raise eyebrows about the rest of his work.

I suspect that many of the effects his research explored are far more subtle than his articles represented them to be. Some of them probably don't exist at all. And I wonder how many retractions we would see if someone did a full audit of Wansink's work. But the norms-and-monitoring account is such a satisfying interpretation of eating behavior that I find it hard to let go of.

It's irritating.

REFERENCES

This should give the current list of retracted papers by Brian Wansink.

You can find many relevant posts on Retraction Watch by searching Wansink's name. But here is the one where he denies any wrongdoing.

The meta-analysis on the influence of plate size is here, finding an effect if the consumer self-serves and a stronger effect when people didn't realize they were participating in a food study: Holden, S. S., Zlatevska, N., & Dubelaar, C. (2016). Whether smaller plates reduce consumption depends on who’s serving and who’s looking: a meta-analysis. Journal of the Association for Consumer Research, 1(1), 134-146.

The "predictably irrational" quote is a nod to Dan Ariely's popular book on behavioral economics, Predictably Irrational: The Hidden Forces that Shape Our Decisions. Ariely has also been criticized for sloppy data handling (and potentially manipulating data).

Incidentally, here's a handful of papers that have not been retracted – at least not yet – from Wansink. The first is a paper that summarizes and interprets a lot of the work on eating behavior – both Wansink's and others. The second explores cultural differences in how people decide to end their current meal.

Wansink, B. (2010). From mindless eating to mindlessly eating better. Physiology & Behavior, 100(5), 454–463. https://doi.org/10.1016/j.physbeh.2010.05.003

Wansink, B., Payne, C. R., & Chandon, P. (2007). Internal and External Cues of Meal Cessation: The French Paradox Redux? Obesity, 15(12), 2920–2924. https://doi.org/10.1038/oby.2007.348