Comments on: Tetlock Wisdom http://www.overcomingbias.com/2009/08/tetlock-wisdom.html Overcoming Bias is economist Robin Hanson’s blog, on honesty, signaling, disagreement, forecasting, and the far future. Sat, 11 Feb 2012 22:21:13 +0000 hourly 1 http://wordpress.org/?v=3.1.3 By: Ariel http://www.overcomingbias.com/2009/08/tetlock-wisdom.html#comment-431983 Ariel Thu, 27 Aug 2009 08:02:39 +0000 http://www.overcomingbias.com/?p=19532#comment-431983 If you haven't already come across this then I think you'll find this both interesting and pertinent to the topic of the above post. I have difficulty figuring out by what methods the predictions are really being made, but it does seem that D. Sornette is in fact going out on a limb. This is definitely the the right forum to evaluate this. Dragon-Kings, Black Swans and the Prediction of Crises http://arxiv.org/abs/0907.4290v1 Common group dynamic drives modern epidemics across social, financial and biological domains http://arxiv.org/abs/0907.3600v1 If you haven’t already come across this then I think you’ll find this both interesting and pertinent to the topic of the above post. I have difficulty figuring out by what methods the predictions are really being made, but it does seem that D. Sornette is in fact going out on a limb. This is definitely the the right forum to evaluate this.

Dragon-Kings, Black Swans and the Prediction of Crises
http://arxiv.org/abs/0907.4290v1

Common group dynamic drives modern epidemics across social, financial and biological domains
http://arxiv.org/abs/0907.3600v1

]]> By: Eric Falkenstein http://www.overcomingbias.com/2009/08/tetlock-wisdom.html#comment-431954 Eric Falkenstein Wed, 26 Aug 2009 14:17:32 +0000 http://www.overcomingbias.com/?p=19532#comment-431954 One problem is that if you have a lot of value in your perceived expertise, you don't want to expose yourself to a small number of bets, because your 'estimated value' will have a large standard error compared to your 'true value'. Think Julian Simon and Paul Ehrlich entered in a famous wager in 1980, betting on a mutually agreed upon measure of resource scarcity over the decade leading up to 1990. Ehrlich ultimately lost the bet, and all five commodities that were selected as the basis for the wager have continued to trend downward since 1980. Did Ehrlich change his mind? No. He thinks the timescale was simply too short. He may be right (it's not obviously wrong). There are an intrinsically small number of 'factors' related to your expertise, so say you believe in Global Warming, and most bets relevant over the next 5 years are highly correlated, all having a large error component. Alternatively, you may think immigration is great in general, but then dislike all the current immigration plans in practice. Any big policy, in practice, is co-mingled with other policies, and often you are comparing to the hypothetical 'what would have happened if we did not do X'. One solution most people apply is to ignore these issues because they are nonempirical, but clearly this can be disastrous. One problem is that if you have a lot of value in your perceived expertise, you don’t want to expose yourself to a small number of bets, because your ‘estimated value’ will have a large standard error compared to your ‘true value’. Think Julian Simon and Paul Ehrlich entered in a famous wager in 1980, betting on a mutually agreed upon measure of resource scarcity over the decade leading up to 1990. Ehrlich ultimately lost the bet, and all five commodities that were selected as the basis for the wager have continued to trend downward since 1980. Did Ehrlich change his mind? No. He thinks the timescale was simply too short. He may be right (it’s not obviously wrong).

There are an intrinsically small number of ‘factors’ related to your expertise, so say you believe in Global Warming, and most bets relevant over the next 5 years are highly correlated, all having a large error component. Alternatively, you may think immigration is great in general, but then dislike all the current immigration plans in practice. Any big policy, in practice, is co-mingled with other policies, and often you are comparing to the hypothetical ‘what would have happened if we did not do X’.

One solution most people apply is to ignore these issues because they are nonempirical, but clearly this can be disastrous.

]]> By: Robin Hanson http://www.overcomingbias.com/2009/08/tetlock-wisdom.html#comment-431953 Robin Hanson Wed, 26 Aug 2009 14:06:03 +0000 http://www.overcomingbias.com/?p=19532#comment-431953 Sorry; fixed. Sorry; fixed.

]]> By: Robert Koslover http://www.overcomingbias.com/2009/08/tetlock-wisdom.html#comment-431951 Robert Koslover Wed, 26 Aug 2009 13:48:52 +0000 http://www.overcomingbias.com/?p=19532#comment-431951 I guess I'm an exception to the masses here. If I didn't care whether the prognosticators were right or wrong, I wouldn't pay attention to them. In the public fog of information and misinformation, wishy-washy predictions by would-be wisemen seriously reduce the signal to noise ratio. I find it even more annoying when these same noise sources later claim they were "right" even if a review of their actual predictions shows so much ambiguity that they could just as easily be judged "wrong." I would prefer that the public more strongly demand of its wannabe sages that they take a clear and unambiguous stand on any topic in which they claim to have greater prognosticative power than the rest of us unwashed masses. Put up or shut up, I say. It's certainly ok to be wrong <em>some of the time</em>. Those who are wrong more often deserve less respect than those with ojectively better track records. But those pundits who deliberately endeavor to hedge, obfuscate, and manipulate the public's perceptions via faux-scholarship and trickery deserve to be shunned as the worthless con men that they are. I guess I’m an exception to the masses here. If I didn’t care whether the prognosticators were right or wrong, I wouldn’t pay attention to them. In the public fog of information and misinformation, wishy-washy predictions by would-be wisemen seriously reduce the signal to noise ratio. I find it even more annoying when these same noise sources later claim they were “right” even if a review of their actual predictions shows so much ambiguity that they could just as easily be judged “wrong.” I would prefer that the public more strongly demand of its wannabe sages that they take a clear and unambiguous stand on any topic in which they claim to have greater prognosticative power than the rest of us unwashed masses. Put up or shut up, I say. It’s certainly ok to be wrong some of the time. Those who are wrong more often deserve less respect than those with ojectively better track records. But those pundits who deliberately endeavor to hedge, obfuscate, and manipulate the public’s perceptions via faux-scholarship and trickery deserve to be shunned as the worthless con men that they are.

]]> By: Michael Bishop http://www.overcomingbias.com/2009/08/tetlock-wisdom.html#comment-431950 Michael Bishop Wed, 26 Aug 2009 13:27:09 +0000 http://www.overcomingbias.com/?p=19532#comment-431950 Prof. Hanson, the link to crooked timber is broken. Prof. Hanson, the link to crooked timber is broken.

]]> By: Looking for a good forecast? « Knowledge Problem http://www.overcomingbias.com/2009/08/tetlock-wisdom.html#comment-431949 Looking for a good forecast? « Knowledge Problem Wed, 26 Aug 2009 13:17:08 +0000 http://www.overcomingbias.com/?p=19532#comment-431949 [...] response to Tetlock, Robin Hanson wonders whether people don’t care so much about the accuracy of forecasts, but really just want to affiliate with high-status impressive [...] [...] response to Tetlock, Robin Hanson wonders whether people don’t care so much about the accuracy of forecasts, but really just want to affiliate with high-status impressive [...]

]]>