Pedantry And Numeracy In Journalism

Includes: COIN, VZ
by: Felix Salmon

Anthony DeRosa retweeted this photo on Wednesday morning, which came with the caption “Math is difficult for many journalists”. I was genuinely confused: I couldn’t see any math errors in the screenshot. So I asked DeRosa where the error was. He replied:

@felixsalmon @katz not a math error, a more egregious error: not getting the facts straight

— Anthony De Rosa (@AntDeRosa) October 30, 2013

Just as I couldn’t see a math error, I couldn’t see anything remotely egregious. Thus began quite a long Twitter conversation, large parts of which DeRosa Storified for me. I proved very bad at getting my point across in tweets, so I promised to explain everything in this post.

The problem that DeRosa had with the stories about the Norwegian man with bitcoin, it turns out, was that they didn’t agree on exactly how many dollars’ worth of bitcoin he bought back in 2009. Some said $22, some said $26, some said $27. That discrepancy, in and of itself, was proof enough, for DeRosa, that many journalists were committing an “egregious error”.

Now the facts of the story were not in dispute at all. The Norwegian man spent 150 Norwegian krone on bitcoin in 2009 while writing a thesis on encryption, forgot about them, and then, in April 2013, during full bitcoin fever, discovered that his digital wallet contained coins worth some 5 million krone. Nice! In dollar terms, his investment went from being worth about $25 to being worth about $900,000.

But DeRosa wanted to know exactly how much the coins were worth at purchase: if one journalist said $22 and another said $26, then at least one of them, and possibly both, were, in his eyes, clearly wrong. You needed to be looking at multiple versions of the story to even see that there was a disparity here — but that’s exactly what DeRosa was doing. And rather than simply ask why there was a disparity, he decided that the individual journalists were doing something very bad.

It turns out that the reason for the disparity is very simple: the dollar-krone exchange rate fluctuated quite a lot in 2009, and it was unclear exactly when the bitcoins were purchased, so no one knows exactly how much the coins were worth, in dollar terms, when purchased. They might have been worth $22, or they might have been worth $27. Really, it doesn’t make any difference: the man made a profit of well over $850,000 whatever his initial investment was.

But there’s a superficial exactness to numbers that doesn’t exist in words, and so people have a tendency to believe that all numbers are much more precise than in fact they are. If the Labor Department releases a report saying that payrolls rose by 148,000 in September, then a reporter who said that payrolls rose by 150,000 would be considered to have her facts wrong — even though the headline number is only accurate to within 100,000 people either way. The actual number of new jobs could easily be anywhere between 44,000 and 252,000 — and indeed there’s a 5% chance that it’s outside even that large range. But because everybody insists on one hard number, one hard number is what they get.

One of the most important skills in financial journalism is numeracy — having a basic feel for numbers. In this case, the reporters covering the story got the numbers right: they should be applauded for that, rather than having brickbats thrown at them. After all, it’s not hard to find examples of reporters getting numbers very wrong. Consider this story, from the New York Post, under the headline “Verizon increases cell bills 7.1% for 95M customers”:

Verizon (NYSE:VZ) didn’t sign up as many new cell phone customers in the third quarter as Wall Street expected — but it still earned more than forecast as it managed to increase the average bill of its 95.2 million wireless customers by 7.1 percent.

The average Verizon Wireless bill jumped to $155.75 a month as of Sept. 30 from $154.63 last year, the company said Thursday.

Now that is a math error — and evidence of deep innumeracy on the part of the journalist who wrote it, as well as a whole series of editors. If you want to work out exactly what the increase is, in percentage terms, of going from $154.63 to $155.75, then you might need a calculator. But if you were numerate, you would know intuitively that it’s very small, on the order of 1%, and that it’s nowhere near 7%. If you get a result of 7.1%, then that means you’ve pressed a wrong button somewhere, and you should do your sums again.

The problem is that we naturally associate numbers with mathematics, and mathematics with accuracy — and we therefore assume that whenever we see a number, we’re dealing with something which is either right or wrong — just as it was in elementary-school arithmetic. When numbers describe the real world, however, they always have error bars; they’re basically shorthand for a probability distribution. So long as the number that’s printed is plausibly somewhere reasonably likely to be in the fat bit of the distribution, it doesn’t make sense for critics like DeRosa to call it out for being inaccurate. After all, pretty much all numbers are inaccurate, especially if you’re trying to measure something (like the value of a certain number of bitcoins) in terms of something else (like dollars). Journalists should work on the basis of the identity of indiscernibles: so long as the meaning of the story isn’t changed, the exact number being used really doesn’t matter.

Let’s say that you saw various news reports about an event, and that different words were used to describe the weather: some said it was “cold”, others “brisk”, others “frosty”, others “wintry”, and so on. You wouldn’t raise an eyebrow: you’d see that they were all describing the same thing, in slightly different language, and you wouldn’t demand an explanation for the “discrepancy”. Well, numbers in news articles behave like words: they’re trying to describe the state of the world. That’s why the NYT has banned the use of “record” or “largest” unless inflation is taken into account. What matters is not the mathematical relationship between abstract numbers, but rather the state of the world that is being described.

In the case of the bitcoin, there was never any doubt about what was being described, and so the journalism did exactly what it was meant to do. There are far too many real problems with genuinely flawed news articles for critics to start playing “gotcha” whenever they see a couple of numbers which say exactly the same thing, even if they’re not mathematically identical.