Quiz: Fox or Hedgehog?

I read a great book a few months ago that provides insight into the nature of bias and error.  It is Expert Political Judgement, by Philip E. Tetlock. One of Tetlock’s discoveries is that there is a significant correlation between expert prediction accuracy and a cognitive style measure introduced by Isaiah Berlin called "fox" vs "hedgehog". Below the fold I go into more detail on this distinction and present a quiz you can take to determine whether you are Fox or Hedgehog.

Tetlock is Professor of Leadership at UC Berkeley, with a background in psychology and political science.  He ran an experiment for many years from the 80s through the early 2000s in which he invited experts on foreign affairs and political science to make predictions about political and economic trends in countries around the world.  Tetlock tracked and evaluated those predictions, developing a large database of both accurate and inaccurate predictions.  He then analyzed it in various ways to find out how accurate the experts were overall, how they compared to other kinds of predictions, and what psychological and social factors influenced expert accuracy.  These results are what make up the book.

There’s meat here for people coming to the book with a variety of interests and needs, but I will focus on the pragmatic question of what factors influence accuracy, with the hope that learning more about these influences can help us to improve the accuracy of our own predictions and analyses.

One of the psychological measures or metrics which Tetlock found was well correlated with expert accuracy goes back to a distinction introduced by Isaiah Berlin in his book, The Hedgehog and the Fox. I haven’t read that book, but based on Tetlock’s presentation, Berlin distinguished between two cognitive styles to which he gave these colorful names.  The hedgehog is said to know one thing and know it well.  He sees events and trends in terms of his big idea, and aggressively extends it into new realms.  Hedgehogs tend to be confident in the applicability of their fundamental concepts and impatient with those who "do not get it".

Foxes in contrast know many small things which they bring to bear in their analyses in a dynamical and flexible way.  They tend to be uncertain and flexible, "on the other hand" types who are skeptical about their own predictive ability and in fact about the whole enterprise of making predictions in such an intractable realm.

To build suspense, I will not say, yet, which cognitive style was found to be more successful in Tetlock’s experiments.  Instead I will offer Tetlock’s test, from the appendix, along with his scoring methodology used to measure where people stand on the Fox-Hedgehog continuum.  You can take it yourself to decide whether you are Fox or Hedgehog.  In a day or two I will post some of Tetlock’s results as to how the two kinds of experts differed in the success of their predictions.

Here are Tetlock’s questions, which I have converted to a point value, rescaling his weightings which came from factor analysis.  If you agree, give yourself that many points, and if you disagree, give yourself the negative of that many points.  Note that some of the questions have a negative point value, so for those you subtract from your score if you agree, and add to your score (make it more positive) if you disagree.

  1. (7 points) Isaiah Berlin classified intellectuals as hedgehogs or foxes.  The hedgehog knows one bit thing and tries to explain as much as possible within that conceptual framework, whereas the fox knows many small things and is content to improvise explanations on a case-by-case basis.  I place myself towards the hedgehog or fox end of this scale. [Note, I think Tetlock meant to give yourself the +7 points if you think you are a fox, and the -7 if you think you are a hedgehog.]
  2. (-3 points) Scholars are usually at greater risk of exaggerating how complex the world is than they are of underestimating how complex it is.
  3. (-5 points) We are closer than many think to achieving parsimonious explanations of politics
  4. (4 points) I think politics is more cloudlike than clocklike ("cloudlike" meaning inherently unpredictable; "clocklike" meaning perfectly predictable if we have adequate knowledge).
  5. (-5 points) The more common error in decision making is to abandon good ideas too quickly, not to stick with bad ideas too long.
  6. (-2 points) Having clear rules and order at work is essential for success.
  7. (5 points) Even after I have made up my mind about something, I am always eager to consider a different opinion.
  8. (-6 points) I dislike questions that can be answered in many different ways.
  9. (-4 points) I usually make important decisions quickly and confidently.
  10. (5 points) When considering most conflict situations, I can usually see how both sides could be right.
  11. (-3 points) It is annoying to listen to someone who cannot seem to make up his or her mind.
  12. (4 points) I prefer interacting with people whose opinions are very different from my own.
  13. (1 point) When trying to solve a problem I often see so many options that it is confusing.

As noted above, compute your score by adding the point values for the questions where you agree, and subtracting the points (keeping in mind that subtracting a negative means adding) for the questions where you disagree.  Possible scores range from -54 to 54, with negative scores indicating that you are a hedgehog while positive scores mean you are a fox.  The farther the score from zero, the more clearly you fall into the category and the more applicable results based on this cognitive analysis are likely to be for you.

Go ahead and take the test, see if you are a fox or a hedgehog, and I’ll post more shortly describing how these cognitive styles fared in Tetlock’s data.

GD Star Rating
Tagged as:
Trackback URL:
  • Bruce G Charlton

    I can’t do this quiz – because I want to come out a fox, and can guess how to answer the questions to make it happen.

    I would presume that foxes are better at predicting – if they can be bothered to do it. At any rate, it would be hard to be _worse_ at predicting than hedgehogs…

  • genie

    I bet the result is : hogs are more than fox.

  • http://profile.typekey.com/sentience/ Eliezer Yudkowsky

    So am I the only one who looked at this and said, “Wow, the author of this book must be a *huge* hedgehog?” We should classify the whole world of experts along only one dimension, and one end of the dimension is good and the other end is bad?

    In 1905, Einstein published papers explaining the photoelectric effect, Brownian motion, and what we now call special relativity. Was Einstein a hedgehog, meaning that these were all one big idea, or a fox, meaning that they were three little ideas?

    We all know, I think, certain people who qualify as hedgehogs – unfortunate souls insufficiently skilled to discriminate when a concept does or does not apply nicely; who, hence, in the grip of an enthusiasm, apply that concept to everything. Being a hedgehog, a one-meme zombie, is a failure mode worth distinguishing. But to call everyone else “foxes”, and imply that they succeed by going in the opposite direction and having only small ideas, sounds to me outright wrong.

    There’s probably also class of successes worth distinguishing, of competent professionals who know the many hard-won big ideas of their profession – a physicist who knows how to apply both relativity and quantum ideas. Maybe that looks like a “fox” to Tetlock, because of this remarkably silly prejudice that an idea can’t be big and important unless it’s new. Is the atomic hypothesis a little idea? Is calculus a little idea? Look at all the little things that physicists need to know…

    I would say that expertise consists of being able to tell when a concept does or does not apply nicely, and in being able to carve up problems using many big ideas plus many domain-specific little ideas, and let’s not forget that there are medium-sized ideas too.

    If Tetlock has remarkable experimental results mitigating against the obvious suspicion that Tetlock is hedgehogging by dividing the world into foxes and hedgehogs, let’s hear about it.

  • Guy Kahane

    I agree that we would want to know quite a bit more about the psychological credentials of this little questionnaire–e.g. does it have external validation? how does it correlate with other personality traits?

    Some of the questions seem to me to test, not whether one is a fox, but whether one is a Hamlet (or perhaps more accurately, suffering from a ‘weak’ frontal lobe). Must foxes, in Berlin’s sense, be indecisive?

  • http://profile.typekey.com/robinhanson/ Robin Hanson

    Tetlock’s book is a powerful classic, well worth closer attention, but I agree that this particular quiz of his is kind of doofy. Better to focus on the meat of his analysis.

  • http://gondwanaland.com/mlog/2006/11/22/bias-enumeration/ Mike Linksvayer

    Bias enumeration

    Via Chris F. Masse, theres a new blog you should subscribe to, Overcoming Bias. Robin Hanson is blogging there and Im equally excited to see Hal Finney blogging as well. I previously called Finney a great signal-to-noise enhancer (search …

  • Nancy Overman

    Could you direct me to the posting where you “post[ed] more shortly describing how these cognitive styles fared in Tetlock’s data”?

  • http://profile.typekey.com/halfinney/ Hal Finney
  • Elise Weaver

    I looked back at Isaiah Berlin’s essay, and I think he has a different definition of hedgehog than the scoring on Tetlock’s questions would imply.
    Berlin lists Plato, Dostoevsky, and Dante as hedgehogs, for instance.

  • http://derechosalvaje.wordpress.com/ Steven Nickeson

    The warrior/poet Archilochus is said to be the first to write. “The fox knows many things, but the hedgehog knows one big thing.” Note that Archilochus did not write “The fox knows many small things…” nor did Berlin in his essay. (Does the inserted “small” indicate a bias on Tetlock’s part?) That Berlin listed Shakespeare, Goethe, Pushkin and Tolstoy as foxes indicates that he was not concerned with “small” knowledge and his subjects on the fox side of the ledger could be more accurately described by the word “polymath” instead of one who “tends to be uncertain,” etc. Berlin used the Achilochus quote as an entry into an examination of Tolstoy’s theory of history. His categories line up with Pluralism (foxes)– Monism (hedgehogs), or Structuralism (hedgehogs)– Post-Structuralism (foxes). Tetlock is using the line for an entirely different analysis. If his book were not about political predictability, one could conclude (based on the above test) that he was a management consultant or personal coach. His categories are more in line with Jungian personality typology where “Perceiving” describes one who prefers an open ended perspective (Tetlock foxes) while “Judging” describes the one who continually moves toward closure (Tetlock Hedgehogs); in fact some of the above questions read like they were lifted from a Myers-Briggs test. What Achilochus meant by the original no one knows, the line is only a fragment.

  • http://historyaccess.com Bob

    May I be presumptuous enough to mention a modest piece I’ve written about the Berlin zoological society (also quoting Tetlock at some length)? Here’s a link:


  • Pingback: Please fasten your seatbelt and try to beat the market « The Market Monetarist

  • Pingback: Lecture Notes on Model Thinking I | Model Practice

  • Pingback: Who Is Best at Predicting the Future? | Make Better Stuff

  • Pingback: How To See Into The Future, You Sexy Beast | Rolling Alpha

  • Pingback: Pondering the Future | Random Catastrophe