Battle of the election forecasters

Douglas Hibbs is a political scientist whose "bread and peace" model forecasts presidential election votes pretty well from the economy alone, with corrections for wartime.  (I don’t know how to upload graphs to this blog so I’ll point you to some pretty pictures of how the model works for the elections from 1952 through 2004.)

This is interesting in its own right–if elections can be predicted, how do we make sense of fluctuations in the polls–but what I thought would particularly interest the Overcoming Bias community is Hibbs’s discussion, in his recent article, of an article by William Nordhaus that claimed that economic forecasts did not actually work well in 2004.  Nordhaus writes, "the Republican incumbent candidate in 2004 did significantly worse than would be predicted based on economic and political variables such as incumbency and economic performance."  Hibbs, however, makes a convincing case that Nordhaus just looked at some bad models.  Here’s Hibbs’s paper; the discussion of different forecasting models begins on page 5.

As a bonus, here’s an article by Bob Erikson and Chris Wlezien on why the political markets have been inferior to the polls as election predictors.  Erikson and Wlezien write,


Election markets have been praised for their ability to forecast election outcomes, and to forecast better than trial-heat polls. This paper challenges that optimistic assessment of election markets, based on an analysis of Iowa Electronic Market (IEM) data from presidential elections between 1988 and 2004. We argue that it is inappropriate to naively compare market forecasts of an election outcome with exact poll results on the day prices are recorded, that is, market prices reflect forecasts of what will happen on Election Day whereas trial-heat polls register preferences on the day of the poll. We then show that when poll leads are properly discounted, poll-based forecasts outperform vote-share market prices. Moreover, we show that win-projections based on the polls dominate prices from winner-take-all markets. Traders in these markets generally see more uncertainty ahead in the campaign than the polling numbers warrant—in effect, they overestimate the role of election campaigns. Reasons for the performance of the IEM election markets are considered in concluding sections.

GD Star Rating
loading...
Tagged as:
Trackback URL:
  • http://hanson.gmu.edu Robin Hanson

    A key question is when exactly are these properly discounted poll forecasts available to the public? If they are only available with a long delay, then it still makes sense to point people to the market as the best available forecast. If they are available with a short delay, then why don’t some people base their trades on them, and fix market mistakes?

  • http://profile.typekey.com/andrewgelman/ Andrew

    Robin,

    I wouldn’t be surprised if the markets eventually fixed this. That’s why I wrote that the markets “have been” inferior to the polls rather than that they “are” inferior. Perhaps this very blog entry will be what it takes for the market players to realize they can do better.

    It’s been pointed out, however, that this sort of prediction market has a limitation: if the size of the market is large enough for the money to really matter, there are moral hazards of the Black Sox variety (throwing an election to win the bet); if the market is small enough to avoid this problem, then there can be motivation to spend money on the market to manipulate perceptions. I’m sure this is something you’ve thought about in more general context; I know people have talked about it as a fundamental limitation on election prediction markets.

    Regarding Erikson and Wlezien’s paper: I think the point is that some people have been so impressed by the successes that prediction markets have had, that they’ve been too quick to assume some sort of perfection.

  • http://yudkowsky.net/ Eliezer Yudkowsky

    Andrew, if you really believe this, how much are you betting? Or how much did Erikson and Wlezian bet before publishing their paper? That’s always been the primary reason I give more credence to prediction markets than futzable statistics.

    (Incidentally, on one occasion I used this argument on a Peak Oil believer – “Why aren’t you buying oil?” – and she replied “I’ve been buying the dips for a while now, and doing very well for myself.”)

  • http://thesaifhouse.wordpress.com saifedean

    I still remain very skeptical of statistical prediction of politics: Even a blind hen pecks the odd bit of corn every now and then. And if you have thousands of blind hens, one of them is bound to pick a large number of corns. This doesn’t make blind hens the best mechanism of picking corn.

    I’m pretty sure one could go back to 1952 and find a “predictive” equation based on some combinations of trivial variables such as candidates’ height, disposition, favourite color, family size, or favourite sport and “predict” the outcome of each election since then. And if this formula still somehow holds for the next two elections, it hardly makes it the best and most reliable prediction mechanism.

  • http://profile.typekey.com/andrewgelman/ Andrew

    Eliezer,

    Good question. My short answer is that it seems like too much trouble; I’d rather spend my work time writing books, blog entries, etc. Not to say that it wouldn’t be a good idea for me to put my money where my mouth is, it’s just something I haven’t really been in the business of doing.

    Saifedean,

    Follow the links and take a look at the graph and at the papers. The models of Hibbs and Erikson/Wlezien are not the sort you describe.

  • Claudia Piereckdesa

    Mr Hibbs,

    I think the pollsters are not interested in making predictions any more. Their aim is to influence people with numbers and creating the “reality” convenient for their interest. Who is funding the “prediction polls?” I think this is a good question to ask and who are the ones broadcasting these polls? I am pretty shocked with the treatment Ms Clinton is getting from all the media-predictionpolls circuit. I was fed for three days with fragments of Mr Obama speech in Iowa and Ms Clinton Victory speech in NH was cut at the 3rd sentence (by the broadcasting Network, I do not have cable TV)to, instead broadcast Mr Gibbs (Red Skins Coach ) retirement Speech that was old news (his retirement was exhaustively announced on the previous day)and Ms Clinton Victory Speech New News, something just happening was cut short. I think this is meaningful. Is this a bunch of “Media guys” trying to influence our choices by filtering what we can see ? This episode was a shame, and I hope people become aware of this kind of trick! The polls predicting Ms Clinton loosing in NH, could it be a way to influence the results by putting negative pressure over the ones who cast utilitarian votes?

  • http://www.douglas-hibbs.com Douglas Hibbs

    I’ve recently posted an analysis of the implications of my Bread and Peace model for the 2008 US presidential election at my web site: http://www.douglas-hibbs.com