Who Wants Accuracy?

1 Trackback

1. [...] that was some surveillance of the extent to which our experts can make worthwhile predictions. Now Robin Hanson documents one such venture – which alas died stillborn. A well-connected reporter (who I promised I’d [...]

   By Club Troppo » Let’s make sure we remain in ignorance about our experts June 9, 2011 at 9:30 pm

12 Comments

1. komponisto

   June 8, 2011 at 9:30 pm | Reply

   it was killed, supposedly because management decided readers don’t care as much about pundit accuracy as they’d previous thought.

   Of course that need not have been their real reason – perhaps some folks didn’t like the ratings it was giving to their favorite pundits

   These two reasons sound the same; or at least the second reason sounds like a special case of the first.

2. nazgulnarsil

   June 8, 2011 at 11:20 pm | Reply

   Civilization runs on an illusive narrative. No one really wants to destroy this narrative.

3. DK

   June 9, 2011 at 2:16 am | Reply

   I would have thought it was killed after discovering that pundits don’t make concrete forecasts that can be assessed. E.g., one can go back over seven years’ worth of posts at Marginal Revolution and find a dearth of predictions. Do you make many predictions?

4. botogol

   June 9, 2011 at 5:38 am | Reply

   it’s like complaining that America’s Got Talent isn’t very good at picking genuinely talented mega-stars. It’s not about that: it’s about creating a TV programme people like to watch and spend money on, and sells adverts. If the winner also turns out to be successful that’s a welcome bonus.

   newspaper punditry isn’t about accurate forecasts, it’s about creating an entertaining newspaper that people like to read ad spend money on, and sells adverts. If the forecasts turn out to be good, that’s merely a welcome bonus.

5. daedalus2u

   June 9, 2011 at 9:08 am | Reply

   Paul Krugman had a post about a study like this that was actually published and he links to a copy.

   http://krugman.blogs.nytimes.com/2011/05/02/prognosticate-that/

   It is perhaps understandable, given the results, why certain major Washington media organizations would not want to actually look at the data.

   • David C

     June 9, 2011 at 1:44 pm | Reply

     The direct link:
     http://www.hamilton.edu/news/polls/pundit

6. Diane

   June 9, 2011 at 11:41 am | Reply

   Hmmm interesting, society does do strange things and how it functions always changes.

7. nikki_olson

   June 9, 2011 at 1:50 pm | Reply

   Maybe ‘accuracy’ was too difficult to determine for a significant number of cases. And then given the topic at hand, perhaps they were extra careful that the article they were researching was accurate.

8. Chris Koresko

   June 9, 2011 at 7:03 pm | Reply

   If memory serves, Rush Limbaugh says he’s been independently audited and found to have an accuracy of around 0.99 (99%). Of course that’s for factual statements, not predictions. I haven’t investigated the measurement technique and can’t speak for the quality of the result.

   Both Limbaugh and Glenn Beck often boast about the accuracy of certain individual predictions, but of course that’s anecdotal and must suffer from strong selection biases.

   The results of a carefully controlled study would be enlightening. If it included pundits across the political spectrum, it might give some insight into what world-view is closest to the truth.

   • Joshua Zelinsky

     June 9, 2011 at 8:40 pm | Reply

     See the Hamilton study linked above by David. Liberals predict slightly better than conservatives. Lawyers predict worse than non-lawyers. Overall, the total set of examined pundits does no better than chance.

   • Alan

     June 9, 2011 at 9:55 pm | Reply

     See the work of Phil Tetlock on this subject:

     http://tinyurl.com/3btbkg4

9. Alrenous

   June 11, 2011 at 1:44 pm | Reply

   I’m beginning to think this is the problem with philosophy, as well.

   Science is often described as an agreed upon objective method for determining who is right.

   As far as I can tell, the common complaint about philosophical debates being ‘endless’ is true – philosophers haven’t even tried to agree upon a method for coming to agreement. (Thus, it only happens by chance.)

   Could it be because most affiliate with a philosopher to leech status and signal various values, not because they think he might be correct?
   Extending this, could most logic use by philosophers be because logic is fashionable, not because they want logic to constrain their thoughts?

   Lucky that scientists think an objective method is fashionable.