Overcoming Bias

The Complexity Critique

Razib at Gene Expression:

I have always been struck by starkness of human hypocrisy and its incongruity in the face of avowed beliefs. ... Sin is common, and human weakness in the face of contradiction the norm.  Mens' hearts are easily divided, and simultaneously sincere in their inclinations. ... All this leads to the point that I believe far too many of those of us who wish to comprehend human nature scientifically lack a basic grasp of it intuitively. ... Many atheists simply lack a deep understanding of what drives people to be religious, and that our psychological model of those who believe in gods is extremely suspect. The "irrationality" and "contradiction" of human behavior may be rendered far more systematically coherent simply by adding more parameters into the model. ... When I engage with these sorts of issues with readers of Overcoming Bias or Singularitarians my suspicions become even stronger because I see in some individuals an even greater lack of fluency in normal cognition than my own. ... My point is that understanding human nature is not a matter of fitting humanity to our expectations and wishes, but modeling it as it is, whether one thinks that that nature is irrational or not within one's normative framework.

This frustrating critique is frustrating common: "You're wrong because your model is too simple.  But I'm not going to tell you what your model is missing, at least not in a clear enough way to help you improve your model."  Yes of course almost all our models are too simple.  We all know that; what we don't know is exactly what complexities we should be adding to our models.  And for the record I was a teen cultist and my dad and brother were/are church pastors.

For social scientists I think there is actually an advantage in having a less powerful intuitive understanding of human behavior - it helps us notice things that need explaining.  To want to explain particular human behaviors you first need to see them as puzzling, and people with powerful intuitive understandings can predict behavior so well intuitively that they often don't notice behaviors that are at odds with our best theories. 

New Best Game Theory

The latest American Economic Review says lab experiments have crowned a new best game theory:

Experiments on 12 [completely mixed 2 x 2-]games, 6 constant sum games, and 6 nonconstant sum games were run with 12 independent subject groups for each constant sum game and 6 independent subject groups for each nonconstant sum game. Each independent subject group consisted of four players 1 and four players 2, interacting anonymously over 200 periods with random matching. The comparison of the five theories shows that the order of performance from best to worst is as follows: impulse balance equilibrium, payoff-sampling equilibrium, action-sampling equilibrium, quantal response equilibrium, Nash equilibrium.

So what is this winner, impulse balance equilibrium?

Incomplete Analysis

In reading the comments on my variance-induced test bias post, I was reminded of a big bias loophole in social science: judging when an analysis is complete "enough."  We usually have some status quo policies, and some analyses relevant to those policies.  Each analysis tends to favor some possible policies relative to others, but alas most every analysis is incomplete, leaving out relevant considerations. 

Now we do need to assess which analyzes are most relevant to any given policy question, but at least here experts can, when analyses are similar enough, usually bring to bear some relatively "objective" criteria.  When we ask if the relevant analyses are good "enough" to justify action, however, we can usually appeal only to much weaker standards of evaluation. 

Touching Vs. Understanding

On the plane home last week I talked to a sharp Yale historian, and realized we devote far more resources to preserving historical sites, and to making history available via museums, than we do to funding professional historians to make sense of it all.  That reminded me of complaints that NASA spends far more on sending instruments into space to collect data than it does on funding scientists to analyze that data.  In both cases we collect far more data than ever gets carefully analyzed.

Now part of the explanation must be that the public can more easily see historical sites, museums, and space instruments than historians and data analysts.  But that doesn't seem to me a sufficient explanation - I suspect we are also just more interested in touching the past, and in touching space, than in understanding either.  We talk about understanding because that is a modern applause light, but really we just like to touch exotic things.  The more we can touch, the further is our reach, and the more important and powerful we must be.  I wonder how much more this explains.

Added: We have related desires to see art and sport events in person, up close, and to meet and touch celebrities in person. 

Britain Was Too Small

Apparently the relevant unit in the last singularity was Western Europe; Britain was too small to support the industrial revolution by itself.  From the May American Economic Review: 

This paper sets out to test, with a formal CGE model, the role of trade with the New World, and trade itself, in explaining the growth of productivity and income in Britain in the Industrial Revolution era.  We find, to our surprise, that the New World was only very modestly important, even by the 1850s. Had the Americas not existed, or not been discovered, the effects on productivity and income growth would have been perceptible, but the Industrial Revolution would have looked much as it does to us today. There were ready substitutes for the cotton, sugar, corn and timber of the New World in Eastern Europe, the Near East and South Asia.

However, had all trade barriers been substantial - if, say, a victorious France cut off Britain's access to European, African and Asian raw materials and markets - then the history of the Industrial Revolution Britain would have been very different. British incomes per person, instead of rising by 45% between the 1760s and 1850s would have risen by a mere 5%. The total factor productivity growth rate, already a modest 0.4% per year, would have fallen to 0.22% per year.

The magnitude, scale and transforming power of the Industrial Revolution lay in its unification of technological advance with the military power that generated easy British access to the markets of Europe, the Americas, the Near East and the Far East.

Added:  In sum, the unit of the industrial revolution seems to have been Western Europe, so Britain who started it did not gain much relative to the rest of Western Europe, but Western Europe gained more substantially relative to outsiders.

Lazy Lineup Study

Thursday's Nature suggests standard police line-ups may not be so bad:

The traditional US procedure is familiar to any fan of television cop shows. Witnesses are presented with a line-up that includes both the suspect and a number of innocent people, or foils', and are asked to identify the perpetrator. In the early 1990s ... then attorney-general, Janet Reno, invited experts to form a working group to address how this method could be improved. ... The working group's most important recommendation was that line-ups should be conducted in a double-blind fashion, so that neither the witness nor the official overseeing the procedure would know who the suspect was. The group also recommended that the suspect and foils be presented sequentially rather than simultaneously, and that the witness be asked to make a decision after each one rather than waiting until the end. ...

In 2003, the Illinois State Police commissioned its own study to test line-ups under real-world, field conditions ..., with the cooperation of two psychologists and three of the state's police departments ... [they] spent a year conducting some 700 eyewitness identifications. Some of the procedures were non-blind and simultaneous; the rest were double-blind and sequential. Both conditions were a mix of live line-ups and photo arrays. The team found that the double-blind, sequential technique produced higher rates of foil picks - that is, clear errors - and lower rates of suspect picks than the traditional, nonblind line-up. ...

Bounty Slander

A Post article today, Bounties a Bust in Hunt for Al-Qaeda:

Jaber Elbaneh is one of the world's most-wanted terrorism suspects. In 2003, the U.S. government indicted him, posted a $5 million reward for his capture and distributed posters bearing photos of him around the globe.  None of it worked. Elbaneh remains at large, as wanted as ever. ...

Since 1984, the program has handed out $77 million to more than 50 tipsters, according to the State Department.  ... In 2004, Rep. Mark Steven Kirk (R-Ill.) visited Pakistan to assess why Rewards for Justice had generated so little information regarding al-Qaeda's leadership. He discovered that the U.S. Embassy in Islamabad had effectively shut down the program. There was no radio or television advertising. ...

In 2004, Congress passed a law authorizing the State Department to post rewards as high as $50 million apiece -- a provision with bin Laden in mind. Last fall, Rep. Dan Boren (D-Okla.) went further, introducing a bill that would raise the cap to $500 million. The State Department has declined to boost the reward for bin Laden, arguing that more money was unlikely to do any good and would only add to his notoriety.

Let's see, billions spent via ordinary means, and millions offered in bounties, and it is the bounties they blame for Al-Qaeda's notoriety and elusive leaders?  The billions are spent and gone, while the millions in bounties we only lose when they actually work.  How does this suggest we should prefer ordinary means to bounties?  Perhaps this Post comment explains the real objection:

This "price in his head", millions in rewards business has had a stench to it all along. It's evidence of our own raw materialism and reinforces the idea it's our enemies who occupy the moral high ground.

Showing That You Care

My ambitious theory paper, which attempts to explain diverse health behavior puzzles with just a few assumptions, has finally been published in Medical Hypotheses.  (Print copies were mailed today.)  The abstract: 

Human behavior regarding medicine seems strange; assumptions and models that seem workable in other areas seem less so in medicine. Perhaps, we need to rethink the basics. Toward this end, I have collected many puzzling stylized facts about behavior regarding medicine, and have sought a small number of simple assumptions which might together account for as many puzzles as possible.

The puzzles I consider include a willingness to provide more medical than other assistance to associates, a desire to be seen as so providing, support for nation, firm, or family provided medical care, placebo benefits of medicine, a small average health value of additional medical spending relative to other health influences, more interest in public than private signals of medical quality, medical spending as an individual necessity but national luxury, a strong stress-mediated health status correlation, and support for regulating health behaviors of the low status. These phenomena seem widespread across time and cultures.

Morality Is Overrated

Hanging out with moral philosophers last week at Oxford reminded me of the old complaint that economists neglect morality.  Actually, I think the real problem is the reverse!  Let me explain.

Many people advise us on what to do.  Some discuss personal actions, while others suggest how groups could better coordinate.  And, crucially, some advise us on what we should do, while others advise us on how to get what we want. 

At the personal level, parents, teachers, preachers, and activists tend to tell us what is morally right, while friends, mentors, lawyers, doctors, therapists, and financial planners tend to tell us what will achieve our ends.  At the level of social policy, pundits and wonks give a mixture of rationales for their suggestions.  Moral philosophers, for example, tend to emphasize policies we should pick, while economists tend to emphasize policies to better get us what we want.   

All else equal, we may each prefer to do what is right, but when all else is not equal we often allow other considerations to weigh against morality.  After all, morality is only one of the many ends we pursue.  Yes we want to be moral, but we also want other things, and we each choose as if we often care about those other things more than morality.  (Some say moral beliefs directly cause us to be moral even if we don't want that, but I prefer to describe this as a revealed preference for moral ends, i.e., for "wanting" to be moral.) 

Contingent Truth Value

Does allowing prophets, whistle-blowers, and dissidents to tell people truths they don't want to hear help those other people or hurt them?  Today I heard an excellent talk (see slides and paper) by Roland Benabou explaining how it can help or hurt, depending on the situation:

HURT: If your future is likely to be enjoyable, and if before then anticipating your great future gives you enough joy, then if you come across bad news suggesting otherwise you might enjoy your life more overall if you quickly look the other way and forget about it.  Even if later on you realize you are the sort of person who would forget such news, you'd still reasonably guess you had a good chance of an enjoyable future, and you'd enjoy savoring that prospect, at least for a while.  Someone who forced you to pay attention to the bad news could do you a real harm.

HELP: On the other hand, if a group of you worked together to build an enjoyable future, how hard you each worked might depend on the chances you each assigned to your efforts working out well.  Given that you expected other people to avoid looking at bad news, you might also find it in your interest to avoid looking at bad news, so that you were all in an equilibrium where you all avoided bad news.  But for certain parameter values you might all be better off in a different equilibrium where you all expect each other to look at bad news and change your behavior in response.  In this case someone who collected bad news, saved it, and later forced you all to pay attention to the bad news you had tried to forget could upgrade your equilibrium.  This could do you all a favor, a favor you were individually not willing to do for yourselves. 

The value of truth is contingent, and depends on the details of your world and values.  It is not guaranteed.   So honestly demands that my commitment to truth also be contingent.