Paperhouse reads: Bad Science


My dad would bring the works of Stephen Jay Gould along as his holiday reading for our family weeks in France, and one day he showed me a two-page spread comparing two pictures of dots. In one, the dots were scattered about the page; in the other, they were clumped into whorls and clusters. “Which of these”, asked my dad, “do you think has a pattern?” Obviously, I cheated and looked at the captions so I could get the answer right, but all the same my first inclination was to go for one that with the twists and the spirals.

That was the wrong one: any pattern I’d seen was the result of my grasping brain seeking relationships between randomly positioned objects on the page, while the random-looking sprinkled dots on the other image had been generated with a simple rule governing the space between each point. So, I learnt two things about my judgement. First, that I was very very bad at seeing order in absences; second, that I would eagerly interpret a pattern in any number of things that fell close enough together to seem connected. And, according to the text of the essay¹ this illustrated, most people made the same mistake I did. Human beings are ferociously good strange coincidence detectors, and absolutely horrible at interpreting relationships within large quantities of information.

How horrible? Well, if you read Ben Goldacre’s Bad Science column in the Guardian, every week has a new example of either general failure to comprehend research and statistics, or cynical exploitation of this general failure. The book pulls together some of the great narratives of irrationality covered by Goldacre on his blog and for the paper: the great greasy mess of the Durham fish oils trials, the strange power of homeopathy, the depressing momentum of the MMR scare.

And what feels invigorating and entertaining as a weekly debunking takes on a more depressing character as it builds up into a bigger argument about the weakness of the media and the failure of public understanding of science. Christ, it really is dreadful. Newspapers and broadcast outlets routinely distort figures, present corporate press releases as interesting facts, and confound unlikly anecdotes with evidence – partly because the average jouralist is no better than any other average person at understanding figures, and partly because of the brutalising demands of churnalism and the economic need to pull out a striking headline.

Bad Science is an excellent tutorial in recognising the shabby stories and filtering out the nonsense from your news consumption (you’ll never read a report about nutrition without muttering, “Yes, but are they a dietitian?” to yourself). Once you’ve started disgarding the misinformation and the mangled data, though, there’s almost no science coverage left to read: some Saturdays, there’s probably only Goldacre. And even with his sharp writing and smart jokes, it’s undeniably all a bit bleak.

But you don’t just get your faith in journalism smashed out of you. You also get a high-speed course in all the mental distortions that make mistakes like mine with the dots, or Nick Cohen’s with the MMR, so easy to make. And then you get a breezy walkthrough of the ways science has developed to compensate for these crippling freaks of perception: placebo controlled trials, statistical analysis, things so sublimely elegant that once they’re explained it seems extraordinary that anyone ever got anything right without them – and so precisely counter-intuitive, it’s astonishing that people worked them out at all.

For regular Bad Science readers, a lot of the book will feel familiar. But the force and clarity gained by putting everything in the same place is great, and even if you’ve soaked up everything Goldacre’s ever written – every post, tweet and blog comment – you should still buy and read this book, especially now it comes in a new mass-market paperback with the previously-withheld-due-to-legal-proceedings Matthias Rath chapter. (If you’ve already paid out for the first edition, the extra chapter is available to download. Lovely.) There aren’t many things you can buy that will genuinely make you smarter, but by giving you a thorough education in your own – and the media’s – ignorance, this book is worth an ocean of fish oil.

¹ Look, I know it’s pretty shabby to have started out a review of Bad Science with an unsourced anecdote. I think the essay is in Bully For Brontesaurus, and if anyone’s got a copy to hand and can correct my shambling memories, I’d be powerfully grateful.

Self-destruction and self-regulation

Earlier this week, I was blogging about the reporting of suicide. This weekend, Ben Goldacre’s Bad Science column is a much more thorough treatment of the same subject. On the 20 November last year, many UK newspapers carried a story taken from the Press Association about a death by suicide. Complaints were lodged with the PCC against 14 of these papers; 12 complaints were upheld; one of the reports found against was in the Telegraph, and this is the one Goldacre writes about:

“Man cut off own head with chainsaw” was the headline: “A man cut off his head with a chainsaw because he did not want to leave his repossessed home.” What followed this headline was not a news story: far from it. What the Telegraph published was a horrific, comprehensive, explicit, and detailed instruction manual.

In fact this information was so appallingly technical and instructive that after some discussion we have decided that the Guardian will not print it, even in the context of a critique. It gives truly staggering details on exactly what to buy, how to rig it up, how to use it, and even how to make things more comfortable while waiting for death to come.

I’ve read the article: if I was contemplating suicide and looking for a method, I now know everything necessary to copy this example. By the PCC’s own guidelines, it should never have been published. According to the PCC’s judgement, I shouldn’t be able to read it now:

[The Telegraph] suspended the article from its website following the contact from the PCC.

Which is funny, because I took this screengrab today (handbook bits blacked out):

Screengrab 28 March 2009

So, to review this cascade of twattery: the PCC has guidelines on how suicide should be reported. These guidelines were ignored in 12 cases. The PCC was especially critical of the manner in which the Telegraph‘s online article breached the code, and “expected that the situation would not be repeated”. Two months later, the material is still there and still extravagently explicit. Excellent self regulation there. Fearsome and authoritative as ever.

In the comments thread on the Bad Science blog, this was quickly dragged into freedom of speech bickering. “Freedom speech is not a zero sum game”, said one exasperated commentator: “Free speech and freedom of information is not freedom to shout about it.” This story could have been a news-in-brief. It could have excluded all the detailed instruction derived from the coroner’s report. It could have followed The Samaritans’ simple guidelines for reporting suicide in the least damaging way possible.

Not only did it fail on every particular, but the online article goes on to make things astonishingly worse. Have a look on the left at the “related articles” box: if death-by-chainsaw doesn’t appeal, a thoughtful Telegraph sub (or handy algorithm) has picked out five more power-tool and self-destruction related stories. How about making an exit via wild herbs? Hanging? Seriously, your suicide method could be just a click away, and the Telegraph‘s editorial policy is apparently to make sure you’ve got every detail you need to clinch your own fatality.