Book Review

Science Fictions: exposing fraud, bias, negligence and hype in science

by Stuart Ritchie (Bodley Head, 2020)

AND just when I thought it couldn’t get any worse, it does.

Having written an article in the last issue of this journal, questioning the ability of academic psychological research to deliver reliable information (“Is psychology really a science?”), I didn’t expect to find much that surprised me in Stuart Ritchie’s investigation into similar territory, albeit he does it in much more depth and has compelling credentials, as a lecturer at the Institute of Psychiatry, Psychology and Neuroscience at King’s College London. But my jaw continued to drop as I turned the pages. This carefully researched and well-explained, highly readable book shows us that there is much more cause for concern even than thought, not just in psychology research but in medical, biological and social science research, too.

Ritchie begins with the startling finding that people have unrealised powers of psychic precognition, made by a leading psychology professor at Cornell University, Daryl Bem. Indeed, it was this which set Ritchie out on his quest to dig deep into the flaws of scientific methods and practice, with the hope of finding ways for setting them straight again. So stunned was he by the finding, which appeared to demonstrate that undergraduates, having been presented with a list of 40 words and asked to type out as many of them as they could recall, would recall the very ones that were about to be represented to them (ie after they had typed in their own), that he and two other sceptical psychologists re-ran the experiment. They found nothing of the sort. Even worse, when they presented their findings to the very journal that had published Bem’s paper, the editor roundly informed them that the journal had a policy of never publishing replication studies, even if they disconfirmed the original.

At the same time, a paper in Science by a professor in the Netherlands was also causing concern, and that ended with his admission that he had made up all his studies and all his numbers, creating a masterful work of imaginative invention, but definitely not a piece of science.

While psychology, as Ritchie says, is particularly prone to coming up with untrustworthy results, struggling, as it does, to draw understandings from highly complicated human behaviours, other fields have their own catastrophes to be embarrassed about, arguably often more serious because sometimes they can be life or death. One review of over 3,000 papers in three leading medical journals found nearly 400 that dramatically overturned existing medical knowledge – wrongly. These included the ‘finding’ that it is safer to have a caesarean birth for twins; that infants with a parent with a peanut allergy would themselves be at greater risk of one if given peanuts before age three; that cooling a cardiac-arrest patient could increase survival; and that early mobilisation of stroke patients led to better outcomes.

There can be much at stake. Sweden’s celebrated Karolinska Institute, which had trumpeted the pioneering trachea transplantations performed (and published) by one of its surgeons, initially threatened doctors, also at the Institute, who had quickly discovered that the transplanted patients were severely damaged or died. The Institute reported the doctors to the police for looking at the patients’ records without their consent, before having to back down. The Retraction Watch Database, set up by two concerned individuals Ivan Oransky and Adam Marcus, documents retractions of scientific findings dating from the 1970s. It currently exceeds 18,000 retractions – and this is thought to be an underestimate.

However, most spurious findings don’t get retracted because they don’t involve fraud, conflict of interest, forged authorship, misconduct or mistakes that the authors own up to. Among the many that fall into the less reprehensible category of selective use of data is a claim made in Why We Sleep, the popular book written by neuroscientist Matthew Walker, a professor at the University of California, Berkeley: he forcefully asserts that dreadful health and other undesired consequences will ensue if we don’t regularly get eight hours’ shut-eye at night. Writer/researcher Alexey Guzey put the studies to the test and found, among other things, causation being claimed where there was only correlation.

(Indeed, when Joe Griffin reviewed Walker’s book in this journal in 2017, he, too, was so struck by the claim that, in our increasingly sleep deprived lifestyles, six hours’ sleep or fewer on a regular basis greatly increases risk of heart attacks, strokes, high blood pressure, cancer, diabetes, mental illness and more, that he checked the epidemiology to see if these conditions were on the rise. He was reassured to find that, on the contrary, lifespan is lengthening.)

Meanwhile, the classic findings about cognitive dissonance now turn out not to hold water – at least, scientifically. As Ritchie says, “Cognitive dissonance is a remarkably useful concept that makes intuitive sense” – for which we didn’t need scientists.

There is much more to learn about bias, spin, statistical shenanigans, research on one population generalised to another (usually findings on males blithely applied to women and children), finding patterns where none exist, and so-called ‘salami slicing’, where the same study is divided up to create several papers, featuring different elements of the original, which can then be published in various journals, creating the impression of wider support for its results.

Then there are the eye-popping findings that result from drawing big conclusions from small sample sizes. (Claiming a 60 per cent chance of such and such occurring is rather less impressive if you only looked at 10 people than if you looked at 1,000.) Ritchie is not impressed by the excuses: “When I’ve made comments about the low statistical power in scientific seminars, I’ve often heard replies along the lines of ‘my students need to publish papers to be competitive on the job market, and they can’t afford to do large-scale studies’. This is a prime example of well-intentioned individual scientists being systematically encouraged – some would argue, forced – to accept compromises that ultimately render their work unscientific.”

This book is an unnerving read (mitigated by the enjoyable writing style) and Ritchie does end with a chapter on possible ways to “fix science”. He is optimistic. I find I am less so. So it is highly helpful that, in his appendix, he guides us lay people through what we should be looking out for when trying to evaluate the worth of a sexy-sounding scientific paper.

Reviewed by Denise Winn (2020)