Leading bias researcher turns out to be… biased, renounces result

A few days ago, Robin posted on the Edge’s annual question, which this year is about the changing of minds.  One of the participants (a social scientist who undoubtedly knows lots) is Daniel Kahneman.  It’s impossible to overstate Kahneman’s eminence.  He’s unquestionably one of a handful of top researchers ever, and arguably the most important yet alive, on the subjects that make up the theme of this very blog.  In addition to being one of the inventors of the "heuristics and biases" research program, as well as prospect theory, he also won the 2002 "Nobel Prize" in economics. 

Yet he, too, is not immune from motivated error.  A friend and colleague recently forwarded Kahneman’s Edge answer to me.  Apparently, Kahneman himself was so captivated by the lure of a neat theory to handle some difficulties in hedonic experience that he managed to misinterpret the first set of results!

Our hypothesis was that differences in life circumstances would have more impact on this measure than on life satisfaction.  We were so convinced that when we got our first batch of data, comparing teachers in top-rated schools to teachers in inferior schools, we actually misread the results as confirming our hypothesis.  In fact, they showed the opposite: the groups of teachers differed more in their work satisfaction than in their affective experience at work. This was the first of many such findings: income, marital status and education all influence experienced happiness less than satisfaction, and we could show that the difference is not a statistical artifact.  Measuring experienced happiness turned out to be interesting and useful, but not in the way we had expected.  We had simply been wrong. (Emphasis added)

Social scientists, beware.  If this can happen to Daniel Kahneman, it can happen to anyone.

GD Star Rating
Tagged as: , , , , ,
Trackback URL:
  • Caledonian

    If this can happen to Daniel Kahneman, it can happen to anyone.

    Your emphasis is backwards: if this can happen to anyone, it can happen to Daniel Kahneman.

  • Erik

    Um… Isn’t misinterpreting data what social science is all about?

  • Kevin Dick

    Back in 1990, I had the honor of a taking a class from Amos Tversky, Kahneman’s frequent research partner who had the misfortune to pass away before the 2002 Nobel was awarded.

    His observation, which I assume is shared by Khaneman, is that cognitive biases are similar to perceptual biases. Just because you know about them, doesn’t mean you can easily correct for them. Psychologists know that distance perception is affected by all sorts of factors such as atmospheric clarity, presence of reference objects, etc. That doesn’t mean a psychologist is any better judge of distance than anyone else. It’s just the way everyone’s brain works.

    You need a ruler to do better. The scientific method is like that ruler for cognitive bias (though imperfect). This single idea has always stuck with me and is probably the most significant factor in my desire to read this blog.

  • briarandbramble

    The phrase “renounces result” in the title of this post is inaccurate and misleading.

  • I have much respect for Kahnemann, but for some reason he is pushing
    a seriously distorted line that is based on one of the most obvious and
    flagrant of errors in statistics, confounding cross-section with time-series
    results. He has been supporting the arguments of recent distributors of the
    Gallup survey results around the world that find a positive relation between
    income and reported happiness (and satisfaction, they are strongly correlated),
    as supposedly showing that income does cause happiness after all.

    This has been trumpeted in lots of media and lots of blogs, including Marginal
    Revolution recently, as supposedly disproving the Easterlin Hypothesis, that
    rising income does not necessary lead to rising happiness, when it is the income
    of everybody in a society. People get happier when their incomes rise relative
    to that of others within their own society. To quote Robert Frank: “The happy
    man is he whose wife’s sister’s husband makes less money than he does.”

    So, what is the deal here? Easterlin’s findings are based on extensive studies
    of time-series data, starting with a study of Japan in the 1970s, where income had
    soared at double digit rates for a couple of decades. No increase in overall
    happiness, although the rich tended to be happier than the poor. This has been
    replicated widely, including in the US, where reported happiness actually maxed
    out in 1956. In short, the current set of relations, with higher income countries
    being happier than poorer ones, probably held back a half century ago, with not
    much change in those rankings, a kind of global version of the relative income
    effect that Easterlin found within societies, the rich are happier than the poor,
    but only because they are above the poor. I have suggested elsewhere that there
    may be other factors at work in these findings as well, such as pride in national
    power, which is correlated strongly with income, and so forth.

    I cannot resist adding a really unpleasant point in my view. I have been given to
    understand by high level individuals that the Gallup organization is not making this
    data set generally available. It seems to be available to Kahnemann and a handful of
    others, but not to other reputable and respected researchers who have requested access
    to it. Given the huge publicity that has accompanied this data set, much of misguided
    as I have just pointed out, I find this peculiar and unscientific. Indeed, I consider
    it a travesty and downright reprehensible. I find it unfortunate that the very respected
    Professor Kahnemann appears to be partaking of this regrettable effort to suppress
    scientific reseearch.

  • Doug S.

    I seem to recall reading that absolute income does increase happiness but only to a point; once you’re out of poverty and no longer have to worry about where your next meal is coming from (figuratively speaking), getting richer doesn’t help happiness very much. In “poor countries” there are a lot more people who do worry about whether they’re going to survive the next few months, which seems as though it would have a negative influence on happiness, treadmill or no treadmill.

  • Doug S.,

    The finding you refer to is within a society, based on the relative income effect.
    In Japan in the late 1940s, there were a lot of people on the verge of famine,
    especially right after 1945. Still, not much difference in happiness with 1974.

  • Vargos

    Oh, sweet irony.