I read a great book a few months ago that provides insight into the nature of bias and error. It is Expert Political Judgement, by Philip E. Tetlock. One of Tetlock’s discoveries is that there is a significant correlation between expert prediction accuracy and a cognitive style measure introduced by Isaiah Berlin called "fox" vs "hedgehog". Below the fold I go into more detail on this distinction and present a quiz you can take to determine whether you are Fox or Hedgehog.
I'm a fox ... I have many ideas in my head ... I work a little bit but I'm successful thanks to God ... The success is not about your personality but a way that you are guided in your life to live in until the death ... You might be seen as successful leader or worker but you climb on others backs ... Or you have illness ... Money is not always success ... Being a manager is not always a success
May I be presumptuous enough to mention a modest piece I've written about the Berlin zoological society (also quoting Tetlock at some length)? Here's a link:
The warrior/poet Archilochus is said to be the first to write. "The fox knows many things, but the hedgehog knows one big thing." Note that Archilochus did not write "The fox knows many small things..." nor did Berlin in his essay. (Does the inserted "small" indicate a bias on Tetlock's part?) That Berlin listed Shakespeare, Goethe, Pushkin and Tolstoy as foxes indicates that he was not concerned with "small" knowledge and his subjects on the fox side of the ledger could be more accurately described by the word "polymath" instead of one who "tends to be uncertain," etc. Berlin used the Achilochus quote as an entry into an examination of Tolstoy's theory of history. His categories line up with Pluralism (foxes)-- Monism (hedgehogs), or Structuralism (hedgehogs)-- Post-Structuralism (foxes). Tetlock is using the line for an entirely different analysis. If his book were not about political predictability, one could conclude (based on the above test) that he was a management consultant or personal coach. His categories are more in line with Jungian personality typology where "Perceiving" describes one who prefers an open ended perspective (Tetlock foxes) while "Judging" describes the one who continually moves toward closure (Tetlock Hedgehogs); in fact some of the above questions read like they were lifted from a Myers-Briggs test. What Achilochus meant by the original no one knows, the line is only a fragment.
I looked back at Isaiah Berlin's essay, and I think he has a different definition of hedgehog than the scoring on Tetlock's questions would imply.Berlin lists Plato, Dostoevsky, and Dante as hedgehogs, for instance.
Via Chris F. Masse, theres a new blog you should subscribe to, Overcoming Bias. Robin Hanson is blogging there and Im equally excited to see Hal Finney blogging as well. I previously called Finney a great signal-to-noise enhancer (search ...
Tetlock's book is a powerful classic, well worth closer attention, but I agree that this particular quiz of his is kind of doofy. Better to focus on the meat of his analysis.
I agree that we would want to know quite a bit more about the psychological credentials of this little questionnaire--e.g. does it have external validation? how does it correlate with other personality traits?
Some of the questions seem to me to test, not whether one is a fox, but whether one is a Hamlet (or perhaps more accurately, suffering from a 'weak' frontal lobe). Must foxes, in Berlin's sense, be indecisive?
So am I the only one who looked at this and said, "Wow, the author of this book must be a *huge* hedgehog?" We should classify the whole world of experts along only one dimension, and one end of the dimension is good and the other end is bad?
In 1905, Einstein published papers explaining the photoelectric effect, Brownian motion, and what we now call special relativity. Was Einstein a hedgehog, meaning that these were all one big idea, or a fox, meaning that they were three little ideas?
We all know, I think, certain people who qualify as hedgehogs - unfortunate souls insufficiently skilled to discriminate when a concept does or does not apply nicely; who, hence, in the grip of an enthusiasm, apply that concept to everything. Being a hedgehog, a one-meme zombie, is a failure mode worth distinguishing. But to call everyone else "foxes", and imply that they succeed by going in the opposite direction and having only small ideas, sounds to me outright wrong.
There's probably also class of successes worth distinguishing, of competent professionals who know the many hard-won big ideas of their profession - a physicist who knows how to apply both relativity and quantum ideas. Maybe that looks like a "fox" to Tetlock, because of this remarkably silly prejudice that an idea can't be big and important unless it's new. Is the atomic hypothesis a little idea? Is calculus a little idea? Look at all the little things that physicists need to know...
I would say that expertise consists of being able to tell when a concept does or does not apply nicely, and in being able to carve up problems using many big ideas plus many domain-specific little ideas, and let's not forget that there are medium-sized ideas too.
If Tetlock has remarkable experimental results mitigating against the obvious suspicion that Tetlock is hedgehogging by dividing the world into foxes and hedgehogs, let's hear about it.
I can't do this quiz - because I want to come out a fox, and can guess how to answer the questions to make it happen.
I would presume that foxes are better at predicting - if they can be bothered to do it. At any rate, it would be hard to be _worse_ at predicting than hedgehogs...
I agree "Small" may seem condescending and unimportant in this context.
Well my score is 44 ... hhh which means I'm a fox ... And I like it ...
I'm a fox ... I have many ideas in my head ... I work a little bit but I'm successful thanks to God ... The success is not about your personality but a way that you are guided in your life to live in until the death ... You might be seen as successful leader or worker but you climb on others backs ... Or you have illness ... Money is not always success ... Being a manager is not always a success
May I be presumptuous enough to mention a modest piece I've written about the Berlin zoological society (also quoting Tetlock at some length)? Here's a link:
http://www.historyaccess.co...
The warrior/poet Archilochus is said to be the first to write. "The fox knows many things, but the hedgehog knows one big thing." Note that Archilochus did not write "The fox knows many small things..." nor did Berlin in his essay. (Does the inserted "small" indicate a bias on Tetlock's part?) That Berlin listed Shakespeare, Goethe, Pushkin and Tolstoy as foxes indicates that he was not concerned with "small" knowledge and his subjects on the fox side of the ledger could be more accurately described by the word "polymath" instead of one who "tends to be uncertain," etc. Berlin used the Achilochus quote as an entry into an examination of Tolstoy's theory of history. His categories line up with Pluralism (foxes)-- Monism (hedgehogs), or Structuralism (hedgehogs)-- Post-Structuralism (foxes). Tetlock is using the line for an entirely different analysis. If his book were not about political predictability, one could conclude (based on the above test) that he was a management consultant or personal coach. His categories are more in line with Jungian personality typology where "Perceiving" describes one who prefers an open ended perspective (Tetlock foxes) while "Judging" describes the one who continually moves toward closure (Tetlock Hedgehogs); in fact some of the above questions read like they were lifted from a Myers-Briggs test. What Achilochus meant by the original no one knows, the line is only a fragment.
I looked back at Isaiah Berlin's essay, and I think he has a different definition of hedgehog than the scoring on Tetlock's questions would imply.Berlin lists Plato, Dostoevsky, and Dante as hedgehogs, for instance.
The follow-up posting is here:
http://www.overcomingbias.c...
Could you direct me to the posting where you "post[ed] more shortly describing how these cognitive styles fared in Tetlock's data"?
Bias enumeration
Via Chris F. Masse, theres a new blog you should subscribe to, Overcoming Bias. Robin Hanson is blogging there and Im equally excited to see Hal Finney blogging as well. I previously called Finney a great signal-to-noise enhancer (search ...
Tetlock's book is a powerful classic, well worth closer attention, but I agree that this particular quiz of his is kind of doofy. Better to focus on the meat of his analysis.
I agree that we would want to know quite a bit more about the psychological credentials of this little questionnaire--e.g. does it have external validation? how does it correlate with other personality traits?
Some of the questions seem to me to test, not whether one is a fox, but whether one is a Hamlet (or perhaps more accurately, suffering from a 'weak' frontal lobe). Must foxes, in Berlin's sense, be indecisive?
So am I the only one who looked at this and said, "Wow, the author of this book must be a *huge* hedgehog?" We should classify the whole world of experts along only one dimension, and one end of the dimension is good and the other end is bad?
In 1905, Einstein published papers explaining the photoelectric effect, Brownian motion, and what we now call special relativity. Was Einstein a hedgehog, meaning that these were all one big idea, or a fox, meaning that they were three little ideas?
We all know, I think, certain people who qualify as hedgehogs - unfortunate souls insufficiently skilled to discriminate when a concept does or does not apply nicely; who, hence, in the grip of an enthusiasm, apply that concept to everything. Being a hedgehog, a one-meme zombie, is a failure mode worth distinguishing. But to call everyone else "foxes", and imply that they succeed by going in the opposite direction and having only small ideas, sounds to me outright wrong.
There's probably also class of successes worth distinguishing, of competent professionals who know the many hard-won big ideas of their profession - a physicist who knows how to apply both relativity and quantum ideas. Maybe that looks like a "fox" to Tetlock, because of this remarkably silly prejudice that an idea can't be big and important unless it's new. Is the atomic hypothesis a little idea? Is calculus a little idea? Look at all the little things that physicists need to know...
I would say that expertise consists of being able to tell when a concept does or does not apply nicely, and in being able to carve up problems using many big ideas plus many domain-specific little ideas, and let's not forget that there are medium-sized ideas too.
If Tetlock has remarkable experimental results mitigating against the obvious suspicion that Tetlock is hedgehogging by dividing the world into foxes and hedgehogs, let's hear about it.
I bet the result is : hogs are more than fox.
I can't do this quiz - because I want to come out a fox, and can guess how to answer the questions to make it happen.
I would presume that foxes are better at predicting - if they can be bothered to do it. At any rate, it would be hard to be _worse_ at predicting than hedgehogs...