This week I witnessed two presentations by New York University professor Bruce Bueno de Mesquita, a noted political scientist and futurist. His latest book is called The Predictioneer’s Game and claims to use complex game theory to predict political and economic events. He claims his predictions have been 90% accurate, which is why the CIA and others pay close attention to them. He never told us exactly how his models work, except to say several times that they are “very complex.”
As he spoke about the world and proffered future events — Iran will not develop a nuclear weapon, Colombia and Venezuela will not go to war — it was clear that Mesquita is a smart man who knows a great deal about international politics. He also is a talented public speaker.
Yet something bothered me. During his talk my buddy Justin Rockefeller (also in the room) texted me, “What do you think?” I replied, “Entertaining but I’m deeply skeptical. Nassim Taleb would have a field day.” He replied, “Yep.”
In Fooled by Randomness, Taleb talks about “why human beings are so prone to mistake dumb luck for consummate skill.” The idea of survivorship bias figures prominently in Taleb’s work. If I play the lottery 100 times, and I win every time, this doesn’t necessarily mean I’ve developed the skill to regularly win the lottery. Someone has to win. We ignore those who lose.
Mesquita wasted no breath acknowledging the improbability of developing a mathematical model that reliably predicts world events. He offered no qualifications on how much of his success might be due to luck and randomness. Instead, he dished predictions with breathtaking arrogance and certainty, returning again and again to his 90% success rate. He never once elucidated how this 90% number got calculated (I predict Hugo Chavez will die, eventually) despite it being the source of his credibility.
It took only a few minutes of Googling to find long, detailed criticisms of Mesquita. You’d think such a body of criticisms would temper his certitude. No sirree.
Surprisingly, people in the room seemed taken by Mesquita. He had spot-on observations, to be sure, about the selfishness of Mother Theresa or the self-interest of the Iranian regime. But why didn’t more people eye his prediction schemes with skepticism?
Charisma, for one. He was entertaining. We are so often bored by speakers that anyone who can dance a gig on stage gets a vote for “keeping us awake.” I hear that. But it is a dangerous heuristic — equating entertainment with substance.
He wasn’t just charismatic; he was an “expert.” In general, people are too deferential to experts who make predictions inasmuch as experts sometimes do no better than laypeople at predicting. In particular, people are too deferential to experts toting fancy credentials (such as a PhD), even if those credentials have little to do with the topic at hand. We should be especially skeptical of experts who feel a need to remind us again and again of their expertise, as Mequita did.
Look, Mequita is more than qualified to riff on current affairs and the state of the world. He has written a dozen plus books on international politics and economics and he is more knowledgeable than me on most of the issues he discussed. But it’s unimpressive to commentate under the vague guise of “complex game theory.” Do analysis and make assertions as an informed pundit, like everyone else, not some mathematically gifted prophet whose models only the CIA understands.
Bottom Line: Nassim Taleb’s popularity notwithstanding, there are still intellectuals who take their knowledge too seriously, confuse luck and randomness with skill and foresight, and pontificate with inappropriate levels of certainty in an uncertain, complex world.
Seth Roberts takes Elizabeth Kolbert to task for putting faith in scientists over science. Seth writes about “practically all science journalists”:
They take the consensus view too seriously. In case after case — so many that it’s hard not to draw sweeping conclusions — the consensus view about difficult topics is more fragile than an outsider would ever guess. It’s not necessarily wrong, just less certain.