Justin Wolfers: Judging by the pre-vote polls and prediction markets, the Democratic primary in New Hampshire created one of the most surprising upsets in U.S. political history. Many take this to show prediction markets have failed.
"If Intrade overreacts on a regular basis, and not just on occasion, then this suggests there's an easy way to make some money, as well as starting to correct this at the same time."
This argument gets used a lot. However, the people who know better may not have enough money or want to risk enough money to actually correct the odds.
How much risk is it? Suppose that the market is agreed that a candidate has a 10% chance but I have looked at all the available information very carefully and I believe it's an 80% chance based on that information. How much money should I put on it? Say I invest $50,000 in this candidate and tomorrow an old girlfriend shows up who talks about his sexual fetishes? i couldn't have predicted that, and I'm probably out $50,000.
This might be a reliable way to make money if you can average it out over hundreds of thousands of bets, but I think the stock market is less risky. If the stock market overreacts on a regular basis, then I don't depend on an unknown event. I win when they overreact provided I could get in place to profit by it.
I don't believe arbitrage has anything to do with accuracy. In fact, the word 'accuracy' in the context of prices is about as useful as the word 'fairness' in the context of taxes (how's that for a troll ?).Arbitrage just implies a momentary difference in assessment.
Peter, that was exactly what I meant by arbitrage. This will improve the market because it will tend to penalize more the one with the more inaccurate price. And it is one of the best ways to improve accuracy because it doesn't depend on a prior knowledge of the true probabilities.
The uses of the word arbitrage here suggest that people are confused about the meaning of that word. The standard meaning refers to exploiting inconsistencies between two financial instruments.Market makers / bookies may occasionally use arbitrage, but mostly use other strategies such as reading supply and demand from the behavior of other traders.
My point is that if the financial markets were consistently significantly off compared to other predictors, surely this would be quickly spotted and exploited? In the same way, anyone discovering they were consistently better at deciding football results than the bookies would break the system very quickly. Or receive a number of job offers....
From my understanding, bookies decide their odds by arbitrage, not by estimating the true odds. Similarly, banks spend huge amounts of effort chasing down arbitrage opportunities; according to my friends in banking, actually playing the market by predicting the future values of stocks is a mugs game.
So there may be a limit to how accurate a betting market can become. There may be a method that is consistently more accurate than the market, as long as it's hard to implement, and the returns are relatively lower than the arbitrage opportunities.
A quick impression here: since elections (like sport events) are filled with emotional and irrational beliefs, the arbitrage opportunities may be higher than in the cold stock market. If this is true, we'd expect the accuracy of the election betting markets to be lower than standard financial markets, as the smart bettors go for arbitrage rather than accuracy. Any data here?
But your general point stands: there will be no easy method that is consistently better than betting markets.
Andrew, granted. My point is that if the financial markets were consistently significantly off compared to other predictors, surely this would be quickly spotted and exploited? In the same way, anyone discovering they were consistently better at deciding football results than the bookies would break the system very quickly. Or receive a number of job offers....
Unknown -- I was thinking the same thing, but these markets are too rare. You would need to catch it reacting irrationally many times to make money.
By the way, in '04, when the mid-day exit polls were released, in-trade also wildly over-reacted and swung Kerry to 70%+. That made the day even more depressing for me when the result finally came out.
Yes, I agree. As Justin put it, the prediction markets are an aggregator of the conventional wisdom. When a probabilistic forecast is "wrong," that's part of the prediction error. With enough experience, perhaps we can assign reasonable uncertainty distributions to prediction-market forecasts.
P.S. Regarding Ben Jones's comment above: Not everyone enjoys betting or wants to put the time/effort into it.
I wonder if the Intrade markets are attracting the right people. As a layman I could bet $50 on one side or another, that wouldn't add much information. If the amount to gain were high enough to persuade Wall Street traders to quit their jobs and really investigate what is most probable, now that would add information. Anyone who hasn't done significant investigation into the matter is just adding noise.
Mike,That's a good point, but it still seems to me that the Iowa results were something the market said ex ante was fairly likely. So why the massive post-Iowa correction? Is 38% to 30% that impressive a point spread when the market was expecting an Obama victory?
We do not claim that prediction markets will always be more accurate that other methods.
This is irrelevant. You cannot compute the accuracy of a method with a single datapoint. The only thing one failure shows is that the method isn't guaranteed to be correct.
Ryan, presumably Obama could have won Iowa, but by a small margin, and in this case his chances of winning the nomination probably would not have went up to 70%.
Again, presumably the main reason it was thought that he would not win the nomination was it Clinton would win it; her third place finish in Iowa was evidence against Clinton's winning the nomination, much stronger evidence than if Obama had won and Clinton had finished a close second, and therefore more evidence that Obama would win the nomination.
Whether these arguments are valid or not, they seem to indicate that there was nothing particularly inconsistent about Intrade's beliefs.
"If Intrade overreacts on a regular basis, and not just on occasion, then this suggests there's an easy way to make some money, as well as starting to correct this at the same time."
This argument gets used a lot. However, the people who know better may not have enough money or want to risk enough money to actually correct the odds.
How much risk is it? Suppose that the market is agreed that a candidate has a 10% chance but I have looked at all the available information very carefully and I believe it's an 80% chance based on that information. How much money should I put on it? Say I invest $50,000 in this candidate and tomorrow an old girlfriend shows up who talks about his sexual fetishes? i couldn't have predicted that, and I'm probably out $50,000.
This might be a reliable way to make money if you can average it out over hundreds of thousands of bets, but I think the stock market is less risky. If the stock market overreacts on a regular basis, then I don't depend on an unknown event. I win when they overreact provided I could get in place to profit by it.
I don't believe arbitrage has anything to do with accuracy. In fact, the word 'accuracy' in the context of prices is about as useful as the word 'fairness' in the context of taxes (how's that for a troll ?).Arbitrage just implies a momentary difference in assessment.
Peter, that was exactly what I meant by arbitrage. This will improve the market because it will tend to penalize more the one with the more inaccurate price. And it is one of the best ways to improve accuracy because it doesn't depend on a prior knowledge of the true probabilities.
The uses of the word arbitrage here suggest that people are confused about the meaning of that word. The standard meaning refers to exploiting inconsistencies between two financial instruments.Market makers / bookies may occasionally use arbitrage, but mostly use other strategies such as reading supply and demand from the behavior of other traders.
I would expect going for arbitrage rather than accuracy would be one of the best ways to improve the accuracy of the market.
Andrew, calibrated probabilistic forecasts already tell us their error rate.
My point is that if the financial markets were consistently significantly off compared to other predictors, surely this would be quickly spotted and exploited? In the same way, anyone discovering they were consistently better at deciding football results than the bookies would break the system very quickly. Or receive a number of job offers....
From my understanding, bookies decide their odds by arbitrage, not by estimating the true odds. Similarly, banks spend huge amounts of effort chasing down arbitrage opportunities; according to my friends in banking, actually playing the market by predicting the future values of stocks is a mugs game.
So there may be a limit to how accurate a betting market can become. There may be a method that is consistently more accurate than the market, as long as it's hard to implement, and the returns are relatively lower than the arbitrage opportunities.
A quick impression here: since elections (like sport events) are filled with emotional and irrational beliefs, the arbitrage opportunities may be higher than in the cold stock market. If this is true, we'd expect the accuracy of the election betting markets to be lower than standard financial markets, as the smart bettors go for arbitrage rather than accuracy. Any data here?
But your general point stands: there will be no easy method that is consistently better than betting markets.
Andrew, granted. My point is that if the financial markets were consistently significantly off compared to other predictors, surely this would be quickly spotted and exploited? In the same way, anyone discovering they were consistently better at deciding football results than the bookies would break the system very quickly. Or receive a number of job offers....
Unknown -- I was thinking the same thing, but these markets are too rare. You would need to catch it reacting irrationally many times to make money.
By the way, in '04, when the mid-day exit polls were released, in-trade also wildly over-reacted and swung Kerry to 70%+. That made the day even more depressing for me when the result finally came out.
Robin,
Yes, I agree. As Justin put it, the prediction markets are an aggregator of the conventional wisdom. When a probabilistic forecast is "wrong," that's part of the prediction error. With enough experience, perhaps we can assign reasonable uncertainty distributions to prediction-market forecasts.
P.S. Regarding Ben Jones's comment above: Not everyone enjoys betting or wants to put the time/effort into it.
I wonder if the Intrade markets are attracting the right people. As a layman I could bet $50 on one side or another, that wouldn't add much information. If the amount to gain were high enough to persuade Wall Street traders to quit their jobs and really investigate what is most probable, now that would add information. Anyone who hasn't done significant investigation into the matter is just adding noise.
Mike,That's a good point, but it still seems to me that the Iowa results were something the market said ex ante was fairly likely. So why the massive post-Iowa correction? Is 38% to 30% that impressive a point spread when the market was expecting an Obama victory?
Yes; I think the Intrade odds were just about right. Strange/unexpected things sometimes happen, and it's as simple as that.
The most reliable guide to life that I have found so far is that Paul Krugman is always right about everything.
We do not claim that prediction markets will always be more accurate that other methods.
This is irrelevant. You cannot compute the accuracy of a method with a single datapoint. The only thing one failure shows is that the method isn't guaranteed to be correct.
Ryan, presumably Obama could have won Iowa, but by a small margin, and in this case his chances of winning the nomination probably would not have went up to 70%.
Again, presumably the main reason it was thought that he would not win the nomination was it Clinton would win it; her third place finish in Iowa was evidence against Clinton's winning the nomination, much stronger evidence than if Obama had won and Clinton had finished a close second, and therefore more evidence that Obama would win the nomination.
Whether these arguments are valid or not, they seem to indicate that there was nothing particularly inconsistent about Intrade's beliefs.