Unfalsifiability /// The Bullshitter's Loophole
Confirmation Bias Part 4: Metacognition and the moral cost of certainty
What can be asserted without evidence can also be dismissed without evidence. – Christopher Hitchens (2007)
In Parts 1 through 3 we saw how confirmation bias works, committing us to our tribes and turning us into better liars and litigators. In this final installment, we’ll look at the intellectual backlash to invincible claims and how the refusal to think can underwrite some of humanity’s most rotten behavior.
Unfalsifiability: A Guarantee of Confirmation Bias
We know that falsification is often a more powerful tool than confirmation when it comes to uncovering truth. It’s counterintuitive, but falsification keeps us honest, and we tend to be led astray by our instinct to confirm what we’re already thinking.
But what about unfalsifiable belief systems, where encountering contradictory evidence is fundamentally impossible? In these scenarios confirmation bias is not only likely, but inevitable.
Popper and the eclipse
Karl Popper’s beef with unfalsifiability crystallized in 1919 Vienna, and it coincided with a solar eclipse. He dissected some fashionable theories of the day to make his point: Einstein’s, which got things right, and Freud’s and Adler’s, which got things wrong. In the case of Einstein, an eclipse in May of that year provided the perfect opportunity to falsify his general theory of relativity. The theory predicted something weird, precise, and most importantly testable. During this eclipse the sun would be obscured by the moon, revealing stars just next to it, no longer overpowered by the sun’s brightness. That starlight would follow the curvature of spacetime, warped by the sun’s mass, thus making the stars appear to be in the wrong spot. Einstein himself encouraged astronomers to travel to far reaches of the earth to observe the eclipse, for the chance to prove him wrong. The results were announced in November, and reported rather dramatically by the New York Times:1
“This risky business,” Popper might well have thought, “is how science should be done.” Einstein’s theory stuck its neck out and might have been killed, but it survived.
On the other end of the spectrum we had Freudian and Adlerian theories, also all the rage at the time, and much bigger hits at parties. For some reason, Popper chose a man optionally rescuing or drowning a boy as his example. Man saves boy? Textbook case of sublimation, according to Freud. Adler would point out the man’s obvious inferiority complex. Man drowns boy? Classic Oedipal-induced repression, according to Freud. Adler, the less imaginative of the two, would point out the man’s obvious inferiority complex.
I could not think of any human behaviour which could not be interpreted in terms of either theory. It was precisely this fact – that they always fitted, that they were always confirmed – which in the eyes of their admirers constituted the strongest argument in favour of these theories. It began to dawn on me that this apparent strength was in fact their weakness.2
It was this irrefutability he pinpointed as the hallmark of intellectual fraud. For science to be worth a damn, it needs to be testable, falsifiable. Not only were these sorts of untestable theories bullshit, they were sneaky bullshit. “A typical soothsayer’s trick,” he wrote, “to predict things so vaguely that the predictions can hardly fail: that they become irrefutable.”
Russell and the teapot
Over in England, in a decisive take-down of the unfalsifiable, philosopher and mathematician Bertrand Russell devised his now famous celestial teapot thought experiment. It was to be part of an essay commissioned by Illustrated magazine in 1952, just on the heels of Russell’s win of the 1950 Nobel Prize in Literature. When Russell submitted “Is There a God?”, the magazine chose not to run the piece, without any official explanation.
Here is Russell’s famous mic-drop:
If I were to suggest that between the Earth and Mars there is a china teapot revolving about the sun in an elliptical orbit, nobody would be able to disprove my assertion provided I were careful to add that the teapot is too small to be revealed even by our most powerful telescopes. But if I were to go on to say that, since my assertion cannot be disproved, it is intolerable presumption on the part of human reason to doubt it, I should rightly be thought to be talking nonsense.3
Russell obviously had religious zealots in his sights, perpetually annoyed by their lack of good sense. When challenged with proving the existence of God, a believer might counter, as if in a schoolyard: “Yeah, well you can’t prove God doesn’t exist!” Or, as Russell put it, more surgically and with just a hint of mock diplomacy: “Many orthodox people speak as though it were the business of sceptics to disprove received dogmas rather than of dogmatists to prove them. This is, of course, a mistake.”
It’s the same strategy that Popper had exposed, distilled down to its most frustrating form: make a claim that cannot be tested, then demand respect because it cannot be disproved.
God settles a bet
The God of fundamentalist Christianity (He who tends to animate the MAGA coalition most vigorously) isn’t silent when it comes to His own disprovability. That troll of the Old Testament tests the faithful precisely by withholding evidence of His existence. The Book of Job is basically a warning that even speculating about His mysterious methods is a bad, bad, very bad idea. As the story goes, up in heaven some adversary (Satan?) is taking the piss out of God by claiming His followers only fall in line because He does good things for them, typical fair-weather friends. Take away their good fortune and they’ll turn their back on You, says the provocateur. So, basically to win a bet, God decides to screw with one of his most devout followers, Job. First He sends some ruffians to steal all of Job’s donkeys and kill all his servants (ahem, slaves). Then He sets his sheep and shepherds on fire with lightning. Then He dispatches bandits to steal Job’s camels and kill more servants. Then He blows his house down with all his children inside, killing them. Finally, He infects Job with a “loathsome sores” disease, leading him to sit in the dirt, scraping his skin off with a broken piece of pottery, as one does in such circumstances. Job’s wife, who had had it up to here, is all, “God’s a real prick. Tell him to fuck off, then go ahead and die peacefully.” But Job refuses! And, just like that, God won his divine bet.
But the story wasn’t completely over. After all his tribulations, Job just wants to know why God was treating him like shit. He spends 35 chapters demanding an answer, and God does not appreciate this. To finally shut him up, God shows up disguised as a whirlwind (obviously), reminds Job how mysterious He is (got it!), and tells him to stop asking questions. In other words, don’t probe, don’t bother with evidence, don’t try to rationalize Me, and don’t you dare try to disprove Me, silly mortal. It was a divine endorsement of confirmation bias. And, to this day, people still pray to this bullshit.
The Grand Wizard Donald
And this brings us back to the Grand Wizard of Unfalsifiability: Donald Trump. If a story praises him, of course it is true. The existence of negative information is proof of Trump Derangement Syndrome, just noise from the hippies.
There is no possible evidence that could count against Trump, because every institution capable of producing such evidence has been delegitimized or dismantled. The courts are corrupt. The media is the enemy of the people. The scientists are Soros-funded Marxists. See something otherwise? Your eyes are lying to you. This creates a scrutiny-immune world for Mom and her MAGA cohorts. They don’t have to do the work of weighing conflicting evidence because the Fake News label preemptively disqualifies anything that might challenge their tribal standing, threaten their identity, or hurt their feelings.
Metacognition and moral obligation
So if confirmation bias is baked and beaten into us, should we just resign ourselves to it and to all the bullshit it brings along for the ride? In other words, how fucked are we here? One answer seems to be: maybe not completely, at least when we look at ultra-reduced forms of the problem, using simulations, AI agents, and computational models of decision-making.
In straightforward cases of reinforcement learning, some confirmation bias makes sense. It’s inefficient to verify everything that’s already been confirmed to a high degree of confidence. Sometimes it is smart to go ahead and have faith in your priors. A 2021 study found that when several million simulations were run with artificial agents, confirmation bias was indeed a reward-maximizing strategy. But there was a catch: this was true “only if disconfirming feedback is not completely ignored.”4 Similarly, a 2022 paper from Oxford researchers found that confirmation bias was optimal in certain conditions, but its benefits disappear or become detrimental as environments stabilize and information becomes more reliable.5
So, what’s the best policy for confirmation bias overall? It really just depends. An obvious improvement would be a system that adjusts its bias level based on context. In other words, such systems might need metacognition.
In a 2021 experiment, Max Rollwage and Stephen Fleming used simulation-based modeling with agents of varying levels of “metacognitive insight”. Some agents were perfectly objective with no confirmation bias whatsoever, adapting to be statistically optimal as new evidence arrives - Bayesian belief updaters.6 Other agents adjusted how selectively they processed information according to their metacognition efficiency, which researchers could dial up or down:
We find that the behavioural consequences of selective information processing are systematically affected by agents’ introspective abilities. Strikingly, we find that selective information processing can even improve decision-making when compared with unbiased evidence accumulation, as long as it is accompanied by good metacognition.7
In other words, agents with some confirmation bias combined with strong self-monitoring outperformed the more objective, neutral agents. Nuance matters, and nuance is one of the few things Trump has never been accused of.
So can these simulations, which tend to rely on extremely simple perceptual tasks (e.g., “Is the grid on the left or the right?”) really teach us anything about the rational and political sensibilities of humans? Rollwage and Fleming looked at that too. In a 2018 study, they (along with Raymond Dolan) found that subjects with radical political views were also less competent at these basic perceptual tasks:
We show that individuals holding radical beliefs (as measured by questionnaires about political attitudes) display a specific impairment in metacognitive sensitivity about low-level perceptual discrimination judgments.8
Some people are not only overzealously confident in their conclusions, they are functionally blind to the possibility that they even could be wrong. This is true in 100%-neutral, as-basic-as-it gets, thoroughly apolitical tasks. There is no tribe watching, no ideology to defend. These folks have an error-detection problem.
We can’t just count on ourselves to naturally do what’s right. If something feels right, so natural that we don’t even have to think about it, that is not only a poor measure of its moral worth, it’s a good sign that things need a bit more scrutiny. Doing the right thing might get us into trouble with our pals, embarrass us, or brand us as disloyal. But overcoming some of our most primal human instincts is, ironically, what makes us most human. The good news is the cure isn’t that complicated; all we need to do is think about our thinking. We need to acknowledge evidence that doesn’t fit neatly into our agenda, and give it a fair hearing anyway. Our inability to stop and calibrate our own programming is the kind of cognitive deficit that can scale up from simple error to catastrophic moral failure.
When Hannah Arendt encountered Adolf Eichmann at his 1961 trial in Jerusalem, she noted,
He merely, to put the matter colloquially, never realized what he was doing. [There was no] diabolical or demonic profundity […] He was not stupid. It was sheer thoughtlessness - something by no means identical with stupidity - that predisposed him to become one of the greatest criminals of that period […] That such remoteness from reality and such thoughtlessness can wreak more havoc than all the evil instincts taken together which, perhaps, are inherent in man - that was, in fact, the lesson one could learn in Jerusalem.9
Consider the failure to stop and think about what you’re doing, and to whom. It makes perfect evolutionary sense to hate the tribe across the river. After all, resources are not infinite, and if anyone’s getting prioritized, it’s friends and family, not strangers. And if they look, dress, eat, talk, and pray differently than us… perfect! They can be singled out or lumped together at will, depending on your mission. And what a blessing for the wannabe dictator, whose most rehearsed trick is whipping the masses – convinced they’re victims with a vague perpetrator – into a focused frenzy. Nothing brings out the desire for a strong leader with a loose consideration for the rules like an imagined boogeyman.
In the mid-1990s Carl Sagan wrote, with his trademark uncanny prescience:
Avoidable human misery is more often caused not so much by stupidity as by ignorance, particularly our ignorance about ourselves. I worry that, especially as the millennium edges nearer, pseudoscience and superstition will seem year by year more tempting, the siren song of unreason more sonorous and attractive. Where have we heard it before? Whenever our ethnic or national prejudices are aroused, in times of scarcity, during challenges to national self-esteem or nerve, when we agonize about our diminished cosmic place and purpose, or when fanaticism is bubbling up around us - then, habits of thought familiar from ages past reach for the controls.10
So here is the question that confirmation bias ultimately forces us to confront: What would it take to change your mind? If the answer is nothing – if every counterexample has a ready-made explanation, if every contrary source is compromised, if the absence of proof is treated as proof – then you are no longer acting rationally. And the tragedy is that this feels, from the inside, like the most rigorous thinking of all. The believer experiences their wall of explanations as evidence of how carefully they’ve considered the issues. They have an answer for everything. It never occurs to them that this is precisely their problem. And maybe that’s what it’s like to be a MAGA.
Epilogue: A unicorn chaser by Tim Minchin
If you’ve made it this far and want a funnier, faster take on everything I’ve spent four articles trying to say, see this riff on confirmation bias by the excellent Tim Minchin.
Take good care out there.
Jonathan
Series Table of Contents:
✓ Part 1: Inconvenient Truths and the Mechanics of Being Sure (My MAGA-ed Mom and a history of confirmation bias)
✓ Part 2: Tribal Loyalty and Con(fidence) Artistry (The naive view of information and two ways confirmation bias secures your seat at the table)
✓ Part 3: Self-Deception and the Argumentative Mind (Deep delusion and our instinct to persuade)
✓ Part 4: Unfalsifiable Bullshit and Moral Obligations (MAGA’s invincibility cheat code and the danger of thoughtlessness)
"Einstein Theory Triumphs," The New York Times, November 10, 1919, p. 17.
Karl Popper, Conjectures and Refutations: The Growth of Scientific Knowledge. Basic Books, 1962.
Bertrand Russell, "Is There a God?" (1952), in The Collected Papers of Bertrand Russell, Volume 11: Last Philosophical Testament, 1943–68, ed. John Slater and Peter Köllner. Routledge, 1997, pp. 543–548.
Tor Tarantola, Tomas Folke, Annika Boldt, Omar Perez, and Benedetto De Martino, "Confirmation Bias Optimizes Reward Learning," bioRxiv preprint (2021).
Germain Lefebvre, Christopher Summerfield, and Rafal Bogacz, "A Normative Account of Confirmation Bias During Reinforcement Learning," Neural Computation 34, no. 2 (2022): 307–337.
According to the authors, “A Bayesian agent keeps track of its graded belief that one or other choice option is correct by calculating the probability of the chosen option, given the evidence.”
Max Rollwage and Stephen Fleming, "Confirmation Bias Is Adaptive When Coupled with Efficient Metacognition," Philosophical Transactions of the Royal Society B 376, no. 1822 (2021): 20200131.
Max Rollwage, Raymond Dolan, and Stephen Fleming, "Metacognitive Failure as a Feature of Those Holding Radical Beliefs," Current Biology 28, no. 24 (2018): 4014–4021.
Hannah Arendt, Eichmann in Jerusalem: A Report on the Banality of Evil. Penguin Books, 2006.
Carl Sagan and Ann Druyan, The Demon-Haunted World: Science as a Candle in the Dark. Ballantine Books, 1997.
— Opening epigraph: Christopher Hitchens, God Is Not Great: How Religion Poisons Everything. Twelve Books, 2007, p. 150.
— VFW GIF: "Donald Trump just said something truly terrifying" CNN, July 25, 2018.