Overcoming Bias

Share this post

Keeping score

www.overcomingbias.com

Keeping score

Eric Schliesser
Jan 28, 2007
Share this post

Keeping score

www.overcomingbias.com

A while back Robin Hanson mentioned that somebody should design a mechanism keep to track of pundits’ ‘scores’. Robin notes that there is no feedback mechanism to help the public figure out how accurate pundits’ predictions have been. Moreover, given the way the media works, it’s not clear they have an incentive to find accurate/reliable pundits, rather than entertaining and/or provocative ones. Currently, ‘pundits’ don’t suffer a reputational cost for their blunders. (Anybody recall weapons of mass destruction?) An academic way to go, of course, is to write a critical journal article (also nice at tenure time). For example, the economist/philosopher Erik Angner has written a very nice paper (2006), ‘Economists as Experts: Overconfidence in theory and practice,’ Journal of Economic Methodology 13(1): 1-24 [Fulltext (subscription required); Penultimate draft] in which he analyzes the (somewhat un-inspiring) track-record of Anders Aslund. (Aslund, you may recall, was the Swedish economists that suggested ‘shock therapy’ when he acted as an advisor to the Russian government between 1991 and 1994.) Of course, few pundits deserve such thorough treatment. So, we still await a nice mechanism to ‘score’ and aggregate pundits’ track-records (in the way, say, Ebay merchants are scored). Given the role pundits & talking-heads play in validating (and creating a narrative for) important public policy decisions, this could perform an important public service role. Of course, there is a catch-22 lurking here because it probably requires ‘experts’ to rate/score the ‘experts.’ But given that various foundations are willing to spend serious money on tracking media bias, why not fund a pundit score-keeping institute? In a future post, I’ll discuss the ‘serious’ business of science–and the need for score-keeping in it.

Share this post

Keeping score

www.overcomingbias.com
Comments
TopNewCommunity

No posts

Ready for more?

© 2023 Robin Hanson
Privacy ∙ Terms ∙ Collection notice
Start WritingGet the app
Substack is the home for great writing