Complex Numbers

The Numbers Don't Lie

From Big Think, an article about how the numbers sometimes lie:

     The “McNamara fallacy”: When data leads to the worst decision  
Don’t make the mistake of blindly following quantitative metrics — whether you’re helping clients or looking for lunch.

Or do they?

In 2014, I took a weekend break to York.

You know what sometimes lies? Anecdotes.

York is a lovely city in the north of the UK, with an ancient cathedral, quaint cobbled roads...

Oh, hell no.

...and an interactive Viking experience.

Is that where you get hung by the ankles and forced to watch your clan murdered and village pillaged while you're upside-down?

The reason for the anecdote is quickly revealed: the author made the mistake of trusting online reviews whilst choosing a restaurant, Skewers, which turned out to have high ratings due to drunks lovingly appreciating their late-night-early-morning fare.

Let's be clear, here: This is not a failure of TripAdvisor. It's not a failure of the food establishment. It's certainly not a failure of the well-meaning drunks. The blame lies entirely on the author. I travel quite a bit, and I prefer to form my own opinions of places I visit, rather than relying on others' (sometimes real, sometimes not) experiences. Sometimes, it sucks, but usually, I'm pleasantly surprised. Either way, I get to write reviews of my own, which may or may not factor into someone else choosing to visit.

The story of Skewers is an example of the McNamara fallacy, and learning about it can help us all (especially the underprepared tourists among us).

In spite of my misgivings about the opening anecdote, I'm willing to read on.

Fortunately, the author again gets quickly to the point, which is how I know I'm reading Big Think and not The New Yorker:

The McNamara fallacy is what occurs when decision-makers rely solely on quantitative metrics while ignoring qualitative factors.

So, like, when you rely only on the studies that claim booze is bad for you, and ignore how good it makes you feel.

The fallacy is named after Robert McNamara, the U.S. Secretary of Defense during the Vietnam War, where his over-reliance on measurable data led to several misguided strategies where considering certain human and contextual elements would have been successful.

I'm no expert on war history, but claiming that different decisions "would have been successful" strikes me as arrogant. I'd weasel out with "might have been successful," instead.

The McNamara fallacy is not saying that using data is bad or that collecting as much information as you can is wasted time.

I'm just leaving this here lest someone be thinking, "What's the point in data, then?"

For example, it’s not uncommon for someone to deeply love a book that has few or no reviews on Goodreads. It’s possible to enjoy a restaurant or a movie in spite of what others say. Data is a great starting point, and a great many idiotic and dangerous things are done when we ignore data, but it doesn’t always make for the best decisions.

I'm very familiar with that assessment, anyway. People are different, and you might love something others hate, or vice-versa. Reviews and the like are aggregations, and blindly following them is roughly the same thing as blindly following the crowd. You can do that if you want, but I don't, because that way, I miss out on the joy of making my own discoveries.

If everyone liked the same things, there'd only be one beer, and that would be a boring world indeed.

I have a much bigger problem with the explicit comparison of "finding a place to eat" and "running a war where people die."

Still, I suspect the basic ideas of the fallacy are sound, and the article goes into other examples and applications thereof, which I don't feel the need to reproduce here.

Fear not, though; it's short, and there are very few actual numbers. So, a solid four stars out of five from me. Your experience may vary.