On a charming and effective introduction to numerical literacy
How to Read Numbers: A Guide to Stats in the News (and Knowing When to Trust Them), by Tom Chivers and David Chivers (2021)
How to Read Numbers: A Guide to Stats in the News (and Knowing When to Trust Them). Tom Chivers and David Chivers, Weidenfeld & Nicolson, 2021
Why write about it
Writing essays pegged to books is an interesting process. You want to say something simple, like “Numbers are important. If you’re not a numbers person, you should read this book.”
But then you veer off into wanting to talk about why numbers are important, and then numerical literacy generally; and that leads you to notions of citizenship and democracy and what you want for young people, and somehow everything has become complicated.
So I think that it’s easiest just to say: if you’re not a numbers person, if your eye skips gladly over charts and graphs and longs for lines of nice, clean, straightforward prose, then please keep on reading.
How I came across it
Back in the early 2000s, when I first started teaching, I encountered a statistician named Herb Spirer. He and his wife, Louise, had done a draft of a book called something like “Data Analysis for Human Rights” that did something wonderful: it set out very, very basic information about how to read and think about and present numerical information that one came across. The information was presented in the context of human rights but relevant to any situation.
Once or twice Herb came to my courses and gave everyone, including me, a simple two-hour workshop. We learned things that are so obvious once they are said, but that one might not have thought of on one’s own, like that terms like “many” or “greatly” or “the majority of” are meaningless if one is trying to gain or share an understanding of facts.
Those who had no quantitative background got guidance in untangling things that should not be compared (apples and oranges) and in becoming comfortable with simple charts and other ways of presenting information.
It was Herb, I think, who pointed me to Edward Tufte’s books on data visualisation, or rather to Tufte’s highlighting of Charles Joseph Minard’s representation of Napoleon's catastrophic march on Moscow and subsequent retreat:
If you haven’t encountered this image before, it’s truly a wonder to study. The line on top goes from left to right and depicts the number of soldiers in the invading army as it progresses towards Moscow; the line below shows the numbers of survivors as the soldiers leave Moscow, from right to left, and attempt to return home. You can see an English translation of the whole on Wikipedia as well.
I’m not sure that the Spirers’ book was ever published, at least not in the form that I saw. But seeing how useful it was for those who encountered it, with or without the workshop, and not only in the context of human rights, I have tended to keep an eye open for other sources that can share things that are basic around numbers and that have outsized impacts if one has never happened to encounter their contents before.
In the middle of the pandemic, when quantitative studies were suddenly at the center of all of our lives, I read a review of How to Read Numbers, or perhaps it was an interview with one of the authors, and it sounded so charming and engaging that I ordered a copy right away, even though at that point I pretty much only read fiction when not endlessly reading the news.
I think that, when I ordered it, I was vaguely thinking of how, if it turned to to be good, I might incorporate it into my courses, never mind that I wasn’t actually teaching any at the time. I didn’t expect that I’d end up reading it for pleasure, one very short chapter a day, or passing it along as often as I did.
About the book
A chart or a table of numbers can be wonderfully interesting. However, if one happens to be among those who find numbers inherently a bit scary, one is likely to be a little more relaxed around an image like the one above.
Fortunately, the Chivers and Chivers book (I believe that they are cousins, the one a science writer, the other a professor of economics), has readers like that in mind. There are no calming pictures of restful books, but the whole is driven by an amiably conversational text, and the numbers are optional.
Almost everything that looks like an equation has been cut out and put into boxes outside the main text; you can read them if you like, but if you don’t it won’t limit your understanding. (p.4) )
The goal of the book is different from the data analysis text, in that the aim is not to help people to parse data they encounter, but rather to help people understand some basics around how quantitative studies are conducted and how their results are represented in the news.
The aim is to have readers be better able to catch obvious errors and to interpret information that is presented without essential context.
…[W]hile journalists are usually clever and (despite the stereotype) well intentioned, they’re not traditionally very good with numbers. That means the numbers you read in the news tend to be wrong. Not always, but often enough that it is wise to be wary.
Fortunately, the ways numbers get misrepresented are often predictable: for instance, they can be cherry-picked, by taking an outlier or using a particular starting point, or by chopping up the data repeatedly until you find something; they can be exaggerated, by using a percentage increase rather than the absolute change; they can be used to suggest causation, when really it’s just a correlation, and many other ways. This book will arm you with the tools you need to spot a few of them. (p.4)
Each of those topics is covered in a short chapter, usually only a few pages long. (The longest is around 9-10 pages; almost all the rest can probably be read in 5 minutes or less.)
As it happened, I was up to speed already on most of those, but read the chapters anyway because the turns of phrase and choices of analogy were pleasing and engaging. These include chapters with titles like “Sample Sizes,” “Biased Samples,” “Statistical Significance,” “Has What We’re Measuring Changed?” and more.
But then there are other topics, some of which were not only new to me, but the kind of thing that seems so blindingly obvious once one is told that it’s hard to remember that one never knew it before.
One of these was the chapter called “Demand for Novelty,” which deals with the practices of academic journals. I don’t do quantitative research, and hadn’t realized that journals in quantitative fields disproportionately choose to publish the results of a study if it shows something unexpected, as opposed to a study that simply adds to the field by examining something that might be true, would be unexpected if true, but turns out in the study not to be true. As Chivers and Chivers put it,
This demand for novelty leads to a fundamental problem in science called publication bias. If 100 studies are carried out into whether psychic abilities are real and, say, ninety-two find that they’re not and eight find that they are, that’s a pretty good indicator that they’re not. But if the journals, looking for novelty, only publish the eight that find a positive result, the world will be led to believe that we can see the future. (p.107)
That was useful, and interesting, especially since the rest of the chapter give some simple tools for figuring out if something like that has shaped what is being reported on some particular topic.
I also needed the five pages on Bayes’ Theorem (if you too don’t know that one, it’s a good reason to read this book or one like it), and the seven pages on Forecasting, and the seven on Collider Bias.
But the one that was most new to me was the chapter on Survivor Bias.
The authors give this example to explain the concept of survivor bias:
In 1944, the US Navy was spending a lot of money, effort and lives in bombing Japanese runways. Their bombers were regularly shot at by enemy fighters and ground fire, and many were being destroyed. They wanted to reinforce their armour plating; but armour plating is heavy, and you don’t want to put it all over an aeroplane if you don’t need to, because that will slow it down, make it less manoeuvrable, and reduce its range and maximum payload. (p.142)
To decide where to put the extra armour, they explain, the decision-makers examined the planes that made it back to see where the most damage was, planning to put the additional armour there. But it didn’t initially occur to them to consider where the damage might have been to the planes that had not made it back. In fact, that turned out to the key piece of information, as the planes that made it back had, almost by definition, not been hit in the places where the damage was most likely to be catastrophic. (p.142)
The concept of survivor bias is that information is provided and examined based on a tiny subset that succeeds at something, without considering essential information about the larger category from which that subset might be drawn.
I’m sharing this level of detail from the contents of the book because one of the odd aspects of being a word person who is a bit shy when it comes to numbers is that often we don’t know how much we don’t know.
And a little basic numerical literacy matters, not just for ourselves for also for how we think about the world.
Chivers and Chivers wrote and published this book in the middle of the COVID-19 pandemic, when policy decisions with immediate life-and-death consequences were being made rapidly on the basis of quantitative studies, and when individuals were daily making choices based on the information that was coming out in the press.
In that context, they are explicit in their introduction that they see being a competent consumer of data in the news as not just a personal issue but also an issue of democracy. Their point is that understanding the information out there shapes the knowledge we use to decide what people or policies to vote for.
In order to understand the world around us, we may not need to be good at maths, but we do need to understand how numbers are made, how they’re used and how they can go wrong, because otherwise we’ll make bad decisions, as individuals and as a society. (p.3)
I’d like to take it further. Being aware of how numbers and quantitative studies are being used to shape decision-making makes us more aware of what issues are being chosen to be studied at all, and, even, of who might be being left out.
It helps us to pay attention. When things make a bit of sense to us, when we know what we’re seeing, our eye is less tempted to slip past, to move on to more familiar territory.
When I read this book, entertaining chapter by entertaining chapter, I had such a strong desire to put it, or any other source of similar information, into the hands of teenagers.
I had a sense that the world would be better if it were standard (and for all I know it is, but it didn’t use to be) for young people to gain this kind of basic numeracy even if their main interests have nothing to do with numbers.
At first, I thought that it was for the sake of the teens themselves, because this kind of basic information is empowering: it makes one feel better equipped in the unsteady world that so many are experiencing just now.
But later I thought: no, it’s not really about individuals that I’m thinking. It’s about society as a whole, how good it will be for all of us if two years from now, five years from now, ten years from now, everyone who is forming their core sense of the world at this moment does so with a little numeric literacy under their belts.
So it may be a little silly, but in truth, one of my happiest memories from the worse times in the pandemic involves some friends who read through this engaging little book with their kids, one small chapter per week, and told me afterwards that everyone had enjoyed it.