The Data Detective: Ten Easy Rules to Make Sense of Statistics – Tim Harford
Have you ever heard that babies are delivered by storks? Statistics actually confirm this to be the case: Countries that have higher stork populations also have more babies than those with smaller stork populations.
Of course, this isn’t really true. But this goes to show how easy it is to present statistics to convince people of something that isn’t true. It’s no wonder people are so leery when they hear statistics being thrown around.
In The Data Detective: Ten Easy Rules to Make Sense of Statistics, economist Tim Harford gives a practical guide on how to cut through the half-truths and truly understand the statistics that we come across.
Rule 1: Search your feelings
As human beings, we are mainly driven by feelings. The first rule, while being the most fundamental one, states that we need to detach our emotional feelings from data when we are looking at it.
Look at data, check out your emotions, and don’t jump into conclusions too quick if your feelings are streaming down your brain!
Rule 5: Get the back story
Trying to understand the story behind the data is as important as the data itself: is the data omitting something relevant? Is the report missing anything in particular? Are all the findings crystal clear?
To make it to the point, this is the author’s proposed exercise in this chapter:
After reading an article or a Facebook post describing some cool finding, just ask yourself how you’d explain it to a friend.
Do you know what the researchers did, and why, whether the research was a shock or exactly what experts would have expected? If not, you’re probably in front of just poor journalism.
Rule 7: Demand transparency when the computer says ‘no’
Statistical analysis on small datasets tend to be easy to assess and audit: we do not need fancy algorithms to draw conclusions. On the other hand and since big data started to shine, other more refined approaches surged.
These advanced algorithms usually bring better results to the table, but at the cost of becoming “black-boxes” that are not easy to interpret. Embrace the hype, but do not take these new methodologies as north stars!
Rule 10: Keep an open mind
A man with a conviction is a hard man to change. Tell him you disagree and he turns away. Show him facts or figures and he questions your sources. Appeal to logic and he fails to see your point.
It is possible to gather and to analyze numbers in ways that help us understand the world. Very often we make mistakes not because the data aren’t available, but because we refuse to accept what they are telling use aware when emotions kick in, and do not feel embarrassed when things need to change, that’s what statistics are for.
Rule 3: Avoid premature enumeration
We always need to be sure about what we are talking about when presenting numbers: if you listen that there’s a prevalence of self-harm among young people, try to understand how self-harm is being defined; or if you read that inequality has soared during the last years, make sure you know what kind of inequality is being measured.
Most of the time, newspaper articles and news will only show you half the truth. It is your job to be curious enough to disentangle the other half.
When my information changes, I alter my conclusions. What do you do, sir?
Rule 4: Step back and enjoy the view
Daily news tend to be alarming and to leave you with a sense of short-term memory information overload. The main point of this rule is to slow down the process, get into the context, and spot the main takeaways correctly.
Avoiding alarmism and advocating for long-term, slow-paced information to better understand the context and to get an improved sense of the trend.
Rule 6: Ask who is missing
Big data is starting to be the new normal: tons and tons of information for which only the collectors know what data is being gathered. It is quite possible, though, that the data is biased, meaning that important assumptions might have been made when results are shown.
Do not let the assumption “N (dataset observations) = All” rule your analysis, and always be aware of who or what might not be considered within the data you are dealing with.
The Golden Rule: Be Curious
The cure for boredom is curiosity,” goes an old saying. “There is no cure for curiosity.” Just so: once we start to peer beneath the surface of things, become aware of the gaps in our knowledge, and treat each question as the path to a better question, we find that curiosity is habit-forming.
Sometimes we need to think like Darrell Huff; there is a place in life for the mean-minded, hard-nosed skepticism that asks, Where’s the trick? Why is this lying bastard lying to me?
But while “I don’t believe it” is sometimes the right starting point when confronted with a surprising statistical claim, it is a lazy and depressing place to finish.
Rule 2: Ponder your personal experience
We tend to be overconfident in regards to explaining what we see with our own eyes, “our truth”, and this usually leads us to look for causalities. Bear in mind that causality might be difficult to explain even with great statistics, but impossible without them.
Therefore, the second rule is to leverage the two different points of view regarding the world in general: the worm’s eye, or the personal experience that one has acquired over the years; and the bird’s eye, or the broad but dry insight we get from stats.
Rule 8: Don’t take statistical bedrock for granted
Corrupted governments and politicians, as historical evidence showed, are able to tweak national statistics numbers in order for them to look the way they want them to look.
This doesn’t mean, though, that statistics organizations are not filled with great independent statisticians/economists that will always thrive to deliver the most accurate figures possible.
Rule 9: Remember that misinformation can be beautiful
Behind every graph, there will probably be someone trying to persuade you on taking a specific action: let it be a politician trying to convince you that her new regulation is working wonderfully, or a coworker showing off the great impact of her last proposed feature.
Don’t forget that we show what we want to show, and that all of us tried to persuade someone at any given point in our lives. There is nothing wrong on being persuaded, and it is absolutely not a problem to change our minds.
Comments ()