Data guys and gals will quote "numbers don't lie!" as if slapping a statistic on something automatically makes it gospel.
Numbers don't lie, but they also don't give the whole picture either.
The same dataset can tell completely different stories depending on who's holding the magnifying glass and what they're looking for.
Consider this simple hypothetical scenario.
A report states that "Employee satisfaction is up 10% this quarter."
On the surface, that's a positive number!
Now, what if the survey only went out to employees who voluntarily opted in, or if it coincided with a major company bonus payout?
I think as a starting point, especially within domains that are more commercially driven, we need to normalize being skeptical of statistics. Because of how easily they can be manipulated and not necessarily due to developing an anti-science pov.
My problem with statistics isn't the math itself. That part is relatively straightforward.
It's the conflict between measuring criteria, mode of collection, and interpretation of results where the real manipulation happens.
The very act of drawing a box around a concept to quantify it inherently introduces subjective bias.
Art Of The Count
The criteria you choose to measure literally determines the story you're going to tell.
Conduct your survey online, and you'll skew toward younger, more tech-savvy demographics.
Use landline phones, and you'll catch more older folks who still answer unknown numbers.
Ask "Do you support tax relief for working families?" versus "Do you support reducing government revenue for social programs?" and watch the results flip on the exact same policy.
A startup announces "200% growth in quarterly revenue!" Sounds impressive, right?
But they actually grew from $10,000 to $30,000, while their established competitor grew from $10 million to $15 million. The startup's percentage looks more impressive, but who's actually winning?
Of course, strategic truth-telling is a non-obvious kind of deception and it happens everywhere, even from an individual level up to institutional narratives.
One could argue that this post itself is a strategic truth-telling because I've skewed all my examples toward cases where statistics mislead and condensed the nuance out of a complex topic.
Narrative Play
On a different note, this is where the notorious "correlation does not imply causation" fallacy also appears.
Just because ice cream sales increase with drowning incidents doesn't mean eating ice cream makes you drown. Both are just tied to summer weather.
These so called correlations are always presented in news headlines as if they prove something meaningful.
Statistics have become the ultimate authority play in modern discourse and should always be taken with a hefty pinch of salt.
Whoever has the most impressive-sounding number wins the argument, regardless of context or methodology.
The heart of the matter is we're living in an age where data is treated as objective truth, but the people collecting, analyzing, and presenting that data are subjective beings with agendas, biases, and incentives.
Perhaps, this could be the place where blockchains and decentralized verification systems could eventually play a role, since they could theoretically provide immutable records of how data was collected and processed.
Without throwing the baby out with the bathwater, as in rejecting all statistical analysis entirely, we need to get comfortable asking uncomfortable questions such as who collected this data?
How?
Why?
What story are they trying to tell, and what story might the same data tell if someone else were holding it?
I think as long as information is power, that distinction between data and truth matters more than we realize.
Thanks for reading!! Share your thoughts below on the comments.