Notes & disclaimers:

• If this app is wildly successful, I may make money from it – I hope you’ll agree that this isn’t spam because I’m an avid reader of OB, we wrote this app after reading OB and related material convinced me that it should exist, and I genuinely believe that some of you will like it.
• We’re still in “closed” beta, and there are some rough edges still to be worked off (in particular, if you’re using an Internet Explorer, consider using a good browser instead – we’re still fighting with IE and don’t expect to be finished for at least a couple of weeks).
• We’re very interested in your feedback – please use the feedback tab on the right of the site.

No reason you couldn’t start a track record of track records to determine which events are reliably predicted by track records.

Nobody would suggest a prediction market for lottery numbers.

Definitely… consider actively managed stock funds; some money managers get lucky and beat the market, but past performance is generally uncorrelated with future results.

My biggest reservation is that I think there would be far too much noise on any scenario where one might expect this to have value. Consider if we decided to use a national NCAA mens basketball bracket tournament to determine who should be the sports commentators. I believe there is some understanding one can bring to predicting basketball game outcomes, and combined with deep knowledge of the teams, this should make one a better analysis of sporting events. But these people will rarely come out at the top of the bracket tournament. Bracket tournaments are often won by people who make unusual predictions based on biased loyalties, which by chance happen to play out in a given year. Other people win by guesswork. There is so much chance in a basketball tournament, that I’m not convinced even repeating this program year after year will reveal who is most likely to perform among the best in the next year’s tournament.

The cost of this noise is false confidence — people claiming to have the best advice because of their track record, when in fact they were just lucky.

But it does pose a bit of a difficulty in cases in which we don’t have enough similar events, an event space with holes as it were. How would you handle that?