Overcoming Bias

Dark Police

Years ago many were concerned that new computer and surveillance techs were driving a loss of privacy.  David Brin once thoughtfully argued that we were better off in a transparent society, as long as the light shined also on those in power, such as the police.  Sadly, it seems "privacy" laws now keep light off the police, even while it shines brightly on the rest of us.  From the Volokh Conspiracy: 

Last month, I linked to a story about someone who was "convicted of violating state wiretapping laws" for "conceal[ing] a camera to videotape a Boston University police sergeant ... during a 2006 political protest." I wrote that this was outrageous, but entirely consistent with a 2001 Massachusetts Supreme Judicial Court decision in Commonwealth v. Hyde, which is based on Massachusetts' extremely broad privacy law. The court there upheld a conviction of a person who had "secretly tape recording statements made by police officers during a routine traffic stop" of himself. ... Now ... the Massachusetts Lawyers Weekly reports:

[Simon Glik] will stand trial on Jan. 29 in Boston Municipal Court on charges of wiretapping, aiding an escape and disturbing the peace for allegedly using his cell phone to record the arrest of a 16-year-old juvenile in a drug case....

Maybe we live in a police state, but thank God its a democratic police state ...

Mandatory Sensitivity Training Fails

Today's Washington Post:

A comprehensive review of 31 years of data from 830 mid-size to large U.S. workplaces found that the kind of diversity training exercises offered at most firms were followed by a 7.5 percent drop in the number of women in management. The number of black, female managers fell by 10 percent, and the number of black men in top positions fell by 12 percent. Similar effects were seen for Latinos and Asians.

The analysis did not find that all diversity training is useless. Rather, it showed that mandatory programs -- often undertaken mainly with an eye to avoiding liability in discrimination lawsuits -- were the problem. When diversity training is voluntary and undertaken to advance a company's business goals, it was associated with increased diversity in management.

... Today, U.S. businesses spend from $200 million to $300 million a year on diversity training, ...  Kalev said many trainers and executives told her they were not surprised by her findings. ... Businesses are responding rationally to the legal environment, since several Supreme Court rulings have held that companies with mandatory diversity training are in a stronger position if they face a discrimination lawsuit.

Imagine we had prediction/decision markets twenty years ago on this - could they have foreseen this outcome, and influenced the court to make different legal choices?

Give juries video recordings of testimony

I just finished serving on a jury in a rape trial.  The accused and the alleged victim had been best friends.  The alleged victim testified that she was forcibly raped whereas the defense attorney suggested that the sex was consensual.  The accused never testified.  The only significant evidence presented by the prosecution was the testimony of the alleged victim.  During cross examination the defense attorney attempted to reduce the credibility of the alleged victim and managed to trip her up on one point.  Everyone on the jury agreed that the case came down to the credibility of the alleged victim.   

It would have been extremely helpful during our deliberations to have had a videotape of the alleged victim’s testimony.  But the judge told us that it was not even possible to have a transcript of such testimony.  All we had to rely on were the handwritten notes we were allowed to take during trial.

The cost of providing the jury with a video recording, audio recording or written transcript would have been trivial, perhaps less than the cost of keeping a man in jail for a single day.  So I can’t imagine that cost is the reason why we the jury were denied such useful information.  Rather I suspect that those Massachusetts politicians or officials who have the power to give juries such information don’t greatly care about the accuracy of jury trials.

Double Or Nothing Lawsuits, Ten Years On

Ten years ago today I posted a short essay on what I still think is a great idea:

Let people risk their lawsuits, double or nothing. ... You would write out a simple complaint, including who hurt you when and how, and then take this complaint to the official lawsuit randomizing office, who would then randomly declare it worthless (50% chance) or double it (50% chance). If your suit were doubled, and you took it to court and won, so that the court said your neighbor caused you $X in damages, they would really owe you twice $X. And if you gambled and lost your neighbor would get a record of this, to defend against your trying to sue again over the same complaint.  More generally, you could keep doubling your suit ... Double or nothing suits should create legal incentives for people to avoid hurting others by small amounts, as well as by large amounts. ... Also, I'd let the person being sued also risk the suit, double or nothing, as long as they showed they were good for the maximum damage amount.

Ten years later, the main change I'd make is to require a maximum damage declaration, with perhaps a fee proportional to that max damage, and let the accused party put money into a pot which also gets doubled if the suit is doubled.   So if someone claimed you hurt them by up to $50, you could deposit $50 and then if the suit doubled you'd have $100 ready to pay if needed.   And since you'd have proven your ability to pay, you could double the suit if you liked. 

I wouldn't be surprised if someone else made a similar suggestion, but I haven't seen any such, nor any interest in my suggestion.   The few reactions I have heard have been surprisingly (to me) negative - people don't seem to want law to discourage small harms.   But if so, why do we want law to discourage large harms?

Flexible Legal Similarity

Under legal precedent, judges are supposed to decide the case in front of them similarly to the way similar past cases have been judged.  The October American Journal of Political Science says students are biased to see cases with decisions they liked as more similar to any given new case.   

We conducted two experiments: the first with undergraduates, the second with undergraduates and law students. Participants in each experiment read a mock newspaper article that described a "target case" involving unlawful discrimination. Embedded in the article was a description of a "source case" cited as legal precedent. Participants in both studies were more likely to find source cases with outcomes that supported their policy views in the target dispute as analogous to that litigation.  ... Legal training did not appear to attenuate motivated perceptions.

This suggests real judges similarly see cases as similar as needed to get the outcome they want. 

Confession Errors

Today's Post suggests we rely too heavily on legal confessions: 

In one experiment, Kassin asked volunteers to perform a challenging task on a computer but warned them not to touch the "Alt" key or risk damaging a computer. Volunteers were told that the computer had been damaged and were asked whether they hit the banned key. In reality, the volunteer did nothing wrong. Most volunteers denied it, but as the initial task they were given was made difficult, they became less sure because they were distracted. When researchers had confederates lie about having seen the volunteers hit the Alt key, the number of people who confessed went up to 100 percent. Every stage of increased pressure led ever larger numbers of volunteers to believe they were really guilty.

Noise in the Courtroom!

The acquitted often walk out court with a huge smile on their face, convinced that any stain on their character has been erased.

But more guilty people get tried and acquitted than the average of the population. So barring a dramatic Perry-Mason-like revelation, the trial is evidence of guilt - noisy evidence, but evidence none the less. It isn't legal or scientific evidence, but it is evidence that a Bayesian should use.

But while employers can often access criminal records of convictions, they are generally barred from finding out about acquittals (especially if the accused take steps to have their arrest expunged); and the potential employee is often allowed to lie if asked directly. This noisy measure is deemed officially unavailable.

Should it be available? And in what way is this noisy measure different from those used in education and in medicine?

Bias Against Torture

In many situations it would be better to impose a punishment of torture than imprisonment.  The fact that the U.S. justice system rejects torture as a punishment is the result of an anti-torture bias.

Torture has two benefits over imprisonment.  It’s cheaper for the state to impose and it doesn’t prevent the criminal from engaging in useful labors (such as parenting and working at a job) for long periods of time.   To determine who should be tortured as opposed to imprisoned we need to consider the benefits to society of imprisonment. 

Prison serves three purposes: deterrence, retribution and incapacitation.  Fear of prison deters many would-be criminals from committing crimes.  Fear of torture, however, could do likewise.  Imprisoning criminals can satisfy victims’ desires for vengeance and so make victims feel better.  Torturing criminals could, however, also satisfy victims’ desires for retribution.  Finally, prison prevents imprisoned criminals from attacking people who are not in prison.  The primary disadvantage of torture is that it doesn’t result in the incapacitation of criminals and so leaves them free to strike again.

Many convicted criminals, however, don’t pose a risk to society.  Men convicted of securities fraud, for example, are frequently barred from the stock market and so their freedom won’t endanger society.  Because of its far lower cost, the U.S. should torture rather than imprison criminals who don’t need to be removed from society.

Some would argue that it’s excessively cruel to torture criminals.  But both prison and torture impose costs on criminals.  Why is one type of cost crueler than the other?  If a convicted criminal is indifferent between receiving a certain type of torture or being imprisoned for a given period of time then why would it be excessively cruel to torture but not to imprison?

In the U.S. many prisoners face a significant chance of being raped by a fellow inmate.  This high chance doesn’t seem to bother many people, and is often the subject of jokes.  Yet our society considers it barbaric for a criminal justice system to deliberately torture criminals in ways that may well impose less physical and emotional costs than rape does.  I find these conflicting moral views about torture and imprisonment to be irrational.

Scientific Evidence, Legal Evidence, Rational Evidence

Suppose that your good friend, the police commissioner, tells you in strictest confidence that the crime kingpin of your city is Wulky Wilkinsen.  As a rationalist, are you licensed to believe this statement?  Put it this way: if you go ahead and mess around with Wulky's teenage daughter, I'd call you foolhardy.  Since it is prudent to act as if Wulky has a substantially higher-than-default probability of being a crime boss, the police commissioner's statement must have been strong Bayesian evidence.

Our legal system will not imprison Wulky on the basis of the police commissioner's statement.  It is not admissible as legal evidence.  Maybe if you locked up every person accused of being a crime boss by a police commissioner, you'd initially catch a lot of crime bosses, plus some people that a police commissioner didn't like.  Power tends to corrupt: over time, you'd catch fewer and fewer real crime bosses (who would go to greater lengths to ensure anonymity) and more and more innocent victims (unrestrained power attracts corruption like honey attracts flies).

This does not mean that the police commissioner's statement is not rational evidence.  It still has a lopsided likelihood ratio, and you'd still be a fool to mess with Wulky's teenager daughter.  But on a social level, in pursuit of a social goal, we deliberately define "legal evidence" to include only particular kinds of evidence, such as the police commissioner's own observations on the night of April 4th.  All legal evidence should ideally be rational evidence, but not the other way around.  We impose special, strong, additional standards before we anoint rational evidence as "legal evidence".

As I write this sentence at 8:33pm, Pacific time, on August 18th 2007, I am wearing white socks.  As a rationalist, are you licensed to believe the previous statement?  Yes.  Could I testify to it in court?  Yes.  Is it a scientific statement?  No, because there is no experiment you can perform yourself to verify it.  Science is made up of generalizations which apply to many particular instances, so that you can run new real-world experiments which test the generalization, and thereby verify for yourself that the generalization is true, without having to trust anyone's authority.  Science is the publicly reproducible knowledge of humankind.

Like a court system, science as a social process is made up of fallible humans.  We want a protected pool of beliefs that are especially reliable.  And we want social rules that encourage the generation of such knowledge.  So we impose special, strong, additional standards before we canonize rational knowledge as "scientific knowledge", adding it to the protected belief pool.

Biased Revenge?

Yesterday Bryan Caplan asked:

What's wrong with revenge?  To be more specific: Suppose X is the most severe morally acceptable punishment for act Y committed by person Z. Suppose that the government fails to do anything about Y. What's wrong if a person personally affected by act Y does X to Z?

In the comments, Matt C looks to bias: 

[Compared to public revenge,] personal revenge is too likely to get the wrong guy, and personal revenge is too likely to be excessive.

Acad Ronin looks to feuds:

"[The rule of law is] the transfer of the duty of revenge to the Queen. It's the officers of the Crown avenging a man's murder, not the man's father or the family. Without law what you have is feud, tangling between themselves, and murder repaying murder down the generations. As we have here. But if the Queen's Officers can be relied on to take revenge for a killing, then the feuding must stop because if you feud against the Queen, it's high treason. That's all. That's all that happens in a law-abiding country: the dead man's family know that the Crown will carry their feud for them. Without it you have bloody chaos."
Chisolm, P.F. 1994. A Famine of Horses. New York: Walker and Co.

If the feud problem itself results from biases, these could be the same explanation.