The Perils of Predicting Politics, or Hari Seldon Was Right

When Hari Seldon, the sort-of protagonist of Isaac Asimov’s Foundation novels, developed a method of forecasting he called psychohistory, he never thought it could predict the future perfectly — any number of fundamentally random events might intervene and muck around with what he thought most likely to happen. Seldon had no illusions that he could predict exactly what would happen — instead, he argued he could compute how likely an event was to occur.

A friend asked me recently to comment on recent debates over whether social science research deserved government support when it has such a bad track record of predicting things. Various people have made various arguments, but here’s what I told my friend: Hari Seldon was right. Predicting election outcomes more than a few months in advance is and probably always will be a fool’s errand. Calculating the probability Obama will win reelection this year, on the other hand, is more manageable. 

But the point goes deeper than that, and I want to make this clear: no serious prediction about anything at all concerns anything but probabilities, because there is nothing in this world that is certain.

Consider physics, a discipline with an excellent track record when it comes to prediction. What is it that physicists predict? They predict probabilities — the probability that a given high-energy atomic collision will produce the Higgs boson, for example. They do not predict that a given collision will produce the Higgs. Likewise, climate scientists might predict the probability that the oceans will rise by four feet or more in the next century — though, to be clear, that probability is very close to 100 percent.

Social science is no different, and the reasons are many. Human beings are not just hard to understand and predict; they are very often actually random, even on important matters such as whom to vote for. Social systems are also complex, meaning that even if the rules are simple, there are so many people interacting with each other that the outcomes may be quite complicated. Take just those two things together, and you have a system that’s hard to predict exactly. Probabilities, yes. Certainty, never.

I used to say, and I stand by this claim, that predicting what Congress will do is like predicting a tree. I can tell you how trees — plural — grow, but I can’t tell you what this particular tree will do. I can tell you things about how people behave, and I can tell you things about how Congress behaves, but is John Boehner actually going to do anything about the Affordable Care Act?

Ask Hari Seldon.