The slow death of prediction wizards

Just before I voted on Tuesday, I checked my inbox and discovered a gem: “The election could be because of this email.”

What a tremendous technological change has occurred in the nearly quarter century since hanging chads and butterfly ballots became all the rage.

For months I’ve been receiving emails telling me that this or that candidate is out of the race if I don’t donate immediately. Joe Biden, Barack Obama, and Nancy Pelosi “pop up” in email threads that require my immediate attention. Kamala Harris needs me!

When I observed on social media that Donald Trump often says in his video fundraising appeals that people should only donate if they can afford it, I received an angry response. I took this from a video in which he, What a surpriselaunches multiple attacks on Harris that a Democratic voter might find unfair? What about January 6th?

Well, I don’t know. I found it a bit inappropriate that Trump told potential donors to feed their families before donating money to his campaign. I was also wondering if it might tell us something interesting about the kind of people who donate money to the Trump campaign, the small donors that Republicans have become more dependent on in recent years, rather than as a character reference for the former president, let alone as an unqualified endorsement of everything he ever said or did.

In recent months everyone has indulged in the parlor game of predicting who will win the election. There are betting markets and models and aggregates that claim to tell you who is up and who is down to within 0.1 percentage points.

The polling industry has had its problems in recent election cycles, although I think the misconduct in the midterm elections is somewhat exaggerated. In a bizarre 2016, the Republican majority nationally was not distributed efficiently enough to maximize the party’s gains in the House, and Republican candidates failed in some tight Senate races that were supposed to appear tighter based on a flood of Republican-leaning polls.

I was one of those who thought there would be a red wave two years ago. But I never expected very many Republicans to win by large margins; I expected many Republicans to win by narrow margins. What we’re really talking about is an electoral wave when a solid majority of close races break one way, and not necessarily a repeat of Richard Nixon’s or Ronald Reagan’s 49-state landslides, which isn’t really possible in the current political climate.

What happened instead was that there were seven competitive races that would decide control of the Senate. The six votes that were decided on election day were split down the middle, while the seventh went to a runoff. Democrats won that runoff by a margin of four to three, taking the Senate from a tie broken by the vice president to a pure Democratic majority of 51 to 49. Which, under Senate rules, is still slightly less than that on most legislation functional control of the chamber.

People are pretty upset that the public polls, with the notable exception of Ann Selzer of Iowa, and the major poll watchers all came in at a more or less undecided election. This is herding, they shouted. Or maybe it’s just what happens in a deeply polarized country when a billion dollars’ worth of advertising bombards seven or more states for months, trying to drive each candidate’s negative views into the stratosphere.

It’s possible that we would get better models if people were more interested in the accuracy and accountability of their models and less in hedging or backing up. (Though I suspect that’s really only true if we paid for it, which is why everyone is always looking for tidbits from selectively published private surveys conducted by people who are genuinely interested in the game.)

The reality is that the market for these things outside of campaigns consists of partisans looking for evidence that their side will win, whatever is found in the kind of close elections we’ve had most of the time since 2000, and a few political junkies. There is a likelihood that a binding assumption will encourage more forecasters to deceive their readers rather than develop more reliable models. It would also increase public skepticism about election results if they deviate from their most trusted predictions. Some of this data simply has limited predictive value.

The best local sports writers root for the home team, but aren’t successful enough to sell you a Super Bowl title for a team that’s closer to 2-15. Maybe this will also happen in partisan journalism, but my model gives a less than 50:50 chance.