Open a popular magazine of your choice, or even the newspaper of record, and you’ll find a lot of fascinating claims seemingly backed by scientific aura. Eat this superfood and you’ll be healthy; do this minor thing every day and you’ll be successful; have governments just slightly change some condition that faces us hapless humans and we’ll change the world.
A few months ago, I called this image a “pretend world,”
with pretend ideals, pretend money and pretend language. A world of quick fix and quick bucks, where the road to success no longer requires hard work, just papering over whatever defects emerge.
An idea about simple and revolutionary solutions to complicated problems seems to have consumed the chattering classes, our media elites, and our political overlords. In the last two years, I’ve stumbled across several engaging books trying to fight back against at least some of the research that underlies this nonsense: Stuart Richie at King’s College London wrote Science Fictions: How Fraud, Bias, Negligence, and Hype Undermine the Search for Truth; his colleague at King’s Bobby Duffy, armed with data from his previous job at the polling firm Ipsos MORI, released Perils of Perceptions: Why We’re Wrong about Nearly Everything; yet another Brit, Tim Harford, published How to Make the World Add Up; and Carl Bergstrom and Jevin West released Calling Bullsh*t: The Art of Scepticism in a Data-Driven World.
The latest of these books specializing in “takedowns of stupid research” to land on my desk is Jesse Singal’s The Quick Fix: Why Fad Psychology Can’t Cure Our Social Ills. It’s a pleasant read, as Singal makes his way across various chapters of psychological research claims that turn out to have hyped their results much beyond what they deserve. Some of the specific examples are repeated from the above books, like Daryl Bem’s “extrasensory perception,” where an established psychology professor in a peer-reviewed article in a top-ranked psychology journal showed that university students can see the future (p < 0.05). Same with Amy Cuddy’s power poses: the claim that sitting and standing in more power-like positions can boost our self-esteem to the point where most perceived social ills (e.g., gender outcome gaps, racial discrimination) go away.
Others I wasn’t aware of, like the hundreds of millions of dollars that the US Army plunged into weak, unproven, and frankly ridiculous projects that tried to prevent posttraumatic stress disorder in veterans. Because unvetted positive-psychology research had shown that the Penn Resilience Program, a twenty-hour course specifically targeting children and adolescents, could maybe reduce the onset of depression and anxiety, its proponents could naturally create an Army-wide program for curing PTSD. We can solve the hardest of mental health problems by merely telling people to be happy and optimistic! Science™.
Another fancy idea is the grit revolution: the vague idea that by boosting the ability to work hard or endure hardships in the present in exchange for future benefits, one could ostensibly revolutionize America. We could close the education and outcome gaps between rich and poor or between racial groups by simply teaching the disadvantaged to embrace grit.
It’s low-hanging fruit for Singal to debunk such laughable research, but in each chapter, he bends over backward to respectfully describe them first before he takes them apart. It usually comes down to a combination of common research faults: one-off results that were hyped to oblivion before it turns out that they don’t replicate. Young researchers, through intentional fraud or statistical incompetency and desperate to make a name for themselves in a cutthroat academy (where new, flashy, and positive results are required for publication in top journals) break the rules of proper scientific engagement, concluding that because a finding seems to hold in a narrow, specified, or lab-generated setting it therefore generalizes to big, flashy, real-world outcomes.
The book is about research method problems in psychology, but what ties the chapters together is the credulous belief with which we accept—even long for—shortcuts and simple solutions to hard problems. That small and tiny changes can have outrageously large and lasting social effects—like that flashing an image of an Israeli flag for milliseconds could meaningfully shift “white-hot political divisions among Israelis,” that just beholding the statue The Thinker could “drive churchgoing folk into the arms of Richard Dawkins.” Quick fixes.
One mistake we make is to assume that the people who shout the loudest about their research must thus be right, or even know what they’re talking about.
Singal’s final chapter is on behavioral economics, or more specifically nudging, and I found my dwindling interest suddenly piqued. The takedowns of previous topics were pretty comprehensive, leading the original studies and their proponents to at best conceding, at worst looking like fools. What was Singal to do with the behemoth of fanciful claims that is behavioral economics?
Apparently nothing. Until that final chapter abruptly ended, I was waiting for the attempt at a juicy takedown—all in vain. After almost forty pages of nudging units, Richard Thaler, how belief in homo economicus is silly, and numerous examples of successful(ish) nudge policies, the best we get is a confession that, like other hypes in a book about quick fixes, nudging is too small to achieve any large purpose its proponents may wield it for. Then again, Singal placed behavioral economics alongside quack science like power posing, extrasensory perception, and mumbo-jumbo psychology. He tied it, implicitly at least, with results that could never be replicated, that contained outright fraud, and were achieved with faulty methods. I should take the win.
The paltry nudge criticism aside, the chapter conclusion still stands: “[Y]ou can’t nudge your way out of policy problems.”
It’s easy to walk away from books like these thinking that an entire (sub)field is garbage, that academia is forlorn, that all research is wrong. That’s not true, and these authors are always very careful to stress that that’s not the thesis they’re advancing. Rather, they’re on a quest to expel the misbehaving deviants and thereby increase the public’s belief in what’s left standing.
What these books teach, embody even, is that skepticism is healthy, that there are many ways in which research and researchers can go wrong (intentionally and unintentionally), and that there are plenty more instances of media outlets or political pundits hyping, exaggerating, hijacking or misrepresenting an already weak or faulty finding.
“Don’t trust; verify,” goes a common adage in the bitcoin world. The rest of us should adopt the same mentality, especially when popular or clearly convenient research findings are broadcasted far and loud.
