use of probabilistic methods for the estimation of risks did just blow up the banking system

Did people who are skeptical of statistical models keep more of their money in the housing/mortgage collapse than those who rely on statistical models? Maybe, but I see little evidence for that claim.

I see a widespread pattern of mistakes that appear to have been committed by a much larger set of people than users of statistical models. Homebuyers and politicians fueled the bubble without using any fancy math.

Availability bias appears to explain more of these mistakes than Taleb’s analysis does. Evidence about housing prices from the U.S. during your lifetime is easier to find and remember than evidence going back a century or covering other countries such as Japan. Figuring out whether an X% drop in home prices would cause your company to collapse isn’t easy, because it depends on the size of the mistakes other companies make. But there are numerous financial panics from which we can derive crude statistical models (although the models may look less convincing than models selected to produce results that look more precise).

People who carefully looked for and evaluated as much relevant evidence as they could saw some chance of the current panic happening, regardless of whether they used intuition or fancy statistical models. Some of them warned of the risk. But it was hard for most people to worry about warnings that had been consistently wrong under all the conditions that were fresh in their minds.

Resisting peer pressure isn’t pleasant. The banker who insisted on a 20% down payment for all mortgages got less business during the bubble and was seen by his colleagues as a burden on the bank and an obstacle to helping customers. The regulator who insisted on a 20% down payment for all mortgages was seen as denying the poor the good investments that were available to the rest of the country, and as an obstacle to home ownership (sometimes better described as home borrowing)(governments think home ownership ought to be encouraged, in spite of (or because of?) its tendency to increase unemployment).

I don’t see how I could have done a good job as a banker or bank regulator in 2004 and 2005 – I would almost certainly have quit due to frustration.

What can be done to prevent bubbles from repeating? If investors study history and other countries more objectively, they’ll do less to fuel the bubbles and will be moderately wealthier as a result, but investors need a good deal of patience and thought to make that effort. Avoiding overconfidence would also help, but much of Taleb’s advice boils down to that, and he more or less admits that few people want to follow this advice.
Overcoming availability bias takes time and effort. Maybe AI will change that someday, but until then it’s hard to hope for more than a modest reduction in the harm done by bubbles.

If a regulation requiring 20% down payments on mortgages could be implemented in a way that is as insulated from politics as the margin rules for stocks, there would be fewer foreclosures after the next housing bubble. But the large companies that are being bailed out have shown they would innovate around any similar restrictions on their leverage.

Taleb also claims:

This absence of "typical" event in Extremistan is what makes prediction markets ludicrous, as they make events look binary.

Prediction markets will fail to answer questions that nobody thinks to ask (does any forecasting method not have this problem?), and will sometimes give the same wrong answers that other methods do. Those are reasons to worry about people becoming overconfident about prediction markets, but not to think that there’s a better alternative. (I don’t think Taleb disagrees, but a careless reading of his essay could easily lead people to think he disagrees more than he actually does).

Prediction markets can be designed to focus on binary outcomes, or they can be designed to produce real-valued predictions (such as "the number of combat deaths in the next 5 years", or to better focus attention on extreme outcomes, the log of that number). If you look past Taleb’s hyperbole, you can see a valid concern that the incentives facing companies such as Intrade are causing them to focus on exciting predictions rather than on producing valuable knowledge.

From the paper:

Results We were unable to identify any randomised controlled trials of parachute intervention.

Conclusions As with many interventions intended to prevent ill health, the effectiveness of parachutes has not been subjected to rigorous evaluation by using randomised controlled trials. Advocates of evidence based medicine have criticised the adoption of interventions evaluated by using only observational data. We think that everyone might benefit if the most radical protagonists of evidence based medicine organised and participated in a double blind, randomised, placebo controlled, crossover trial of the parachute.

There are some interesting comments on the paper here and here.

Couldn’t a pure statistician have also observed the inertia in the ratings? Of course. But there are infinitely many biases, patterns, and anomalies to fish for. And in almost every case, the number-cruncher wouldn’t turn up anything. A psychologist, however, can suggest to the statisticians where to point their high-powered mathematical instruments. "It cuts out dead ends,"

This approach applies to a wide variety of problems, including beating markets.
Not only is it important for investors to avoid dead ends in the sense of failing to find patterns, it’s important to distinguish patterns that are sustained by strong human biases from patterns that will vanish when a modest number of people figure out how to exploit them or patterns that are a byproduct of data that are not random samples from the space of all possible market behavior. Or as Coase is reported to have said, "if you torture the data enough, nature will always confess".

There are a variety of known strategies that seem to work even though many people are aware of them. Most seem to be sustained by some combination of Status Quo Bias, Endowment Effect, and Recency Bias, although the benefits of these strategies seem to diminish over time.

Also, since Robin’s advice to welcome diversity in analysis rather than beliefs seems too abstract for some, here’s how I think of putting it into practice when investing: focus on asking questions that few other people are asking. The more widely discussed a question is, the more likely it is that markets reflect the best answer to it. My best investments have been in companies that few people have heard of, often found by looking through more earnings reports than most investors would be willing to. And one infrequently mentioned result from the cognitive science literature has been more helpful in automating my search for underpriced companies than years of studying what other investors are doing.

This appears to be a fairly robust result that is not just a function of truth being the correct guess where the evidence is weak – it shows up in controlled experiments where subjects have good reason not to assume truth (for example, this paper).  Vrij proposes several explanations for this bias.

• The higher frequency of truthful statements in daily life make cause the availability heuristic to bias our judgment.
• Politeness: there’s more social harm in daily life from mistakenly believing someone to be a liar or from asking someone to prove all claims than there is from mistakenly believing false claims.
• People rely on stereotypes which are less accurate for liars than for truth-tellers [it's unclear why this is a separate explanation from the first].

Since it’s unclear whether these effects make it in your interest to be suspicious, let’s also look at how being suspicious affects others. Increased suspicion may do a little harm to your friends by making them a bit more uncomfortable. But if it spreads, it should also improve political systems by making it a bit harder for people to get away with lies. This suggests that altruists ought to be more comfortable with friend who question their honesty in order to encourage social norms under which political choices are based on more accurate beliefs.

One concern I don’t know how to analyze is how this would affect the kind of social capital that Fukuyama talks about in his book Trust.

This book makes a few good points about what cognitive science tells us about differing concepts associated with the word freedom. But most of the book consists of attempts to explain his opponents’ world view that amount to defending his world view by stereotyping his opponents as simplistic.

Even when I agree that the people he’s criticizing are making mistakes due to framing errors, I find his analysis very implausible. E.g. he explains Bush’s rationalization of Iraqi deaths as "Those killed and maimed don’t count, since they are outside the war frame. Moreover, Bush has done nothing via direct causation to harm any Iraqis and so has not imposed on their freedom". Anyone who bothers to listen to Bush can see a much less stupid rationalization – Bush imagines we’re in a rerun of World War II, where the Forces of Evil have made it inevitable that some innocent people will die, and keeping U.S. hands clean will allow Evil to spread.

Lakoff’s insistence that his opponents are unable to understand indirect, systematic causation is ironic, since he shows no familiarity with most of the relevant science of complex effects of human action (e.g. economics, especially public choice economics).

He devotes only one sentence to what I regard as the biggest single difference between his worldview and his opponents’: his opponents believe in "Behavior as naturally governed by rewards and punishments."

His use of the phrase "idea theft" to describe uses of the word freedom that differ from his use of that word is objectionable both due to the problems with treating ideas as property and with his false implication that his concept of freedom resembles the traditional U.S. concept of freedom (here’s an example of how he rejects important parts of the founders’ worldview: "One of the biggest mistakes of the Enlightenment was to counter this claim with the assumption that morality comes from reason. In fact, morality is grounded in empathy").

If his claims of empathy are more than simply calling his opponents uncaring, then it may help explain his bias toward helping people who are most effective at communicating their emotions. For example, a minimum wage is part of his concept of freedom. People who have their wages increased by a minimum wage law tend to know who they are and often have labor unions to help spread their opinions. Whereas a person whom the minimum wage prevents from getting a job is less likely to see the cause or have a way to tell Lakoff about the resulting harm. (If you doubt the minimum wage causes unemployment, see http://www.nber.org/papers/w12663 for a recent survey of the evidence.)

This is symptomatic of the biggest problem with the book – he assumes political disagreements are the result of framing errors, not differences in understanding of how the world works, and wants to persuade people to frame issues his way rather than to use scientific methods when possible to better measure effects that people disagree about.

The book also contains a number of strange claims where it’s hard to tell whether Lakoff means what he says or is writing carelessly. E.g. "Whenever a case reaches a high court, it is because it does not clearly fit within the established categories of the law." – I doubt he would deny that Hamdi v. Rumsfeld fit clearly within established habeas corpus law.

This is a book which will tempt people to believe that anyone who agrees with Lakoff’s policy advice is ignorant. But people who want to combat Lakoff’s ideology should resist that temptation to stereotype opponents. There are well-educated people (e.g. some behavioral economists) who have more serious arguments for many of the policies Lakoff recommends.

[Cross-posted from Bayesian Investor Blog.]

A for-profit prediction market that tries to both maximize profits and provide valuable information to voters faces conflicting pressures, since my observations of existing markets such as those on Intrade show little evidence of a significant correlation between the issues that create profitable trading volume and the issues on which a democracy needs more knowledge. If maximizing profit doesn’t provide useful evidence that the company is achieving its goal, then it’s unclear why we should think that a profit-oriented form is the best way to achieve that goal.
Automated market makers that transfer money to traders seem to be a promising way to improve the accuracy of market prices, but seem inconsistent with a profit-oriented goal. They look more like an act of charity than a business practice when the goal is to inform voters.
Prediction markets face significant regulatory uncertainties, and governments are typically more lenient about regulating charities than profit-oriented companies. This may help nonprofit prediction markets both because a nonprofit will have less incentive to pressure traders to make unwise or unfair trades (i.e. the concerns behind gambling laws and laws mandating disclosure will be somewhat weaker), and because a nonprofit is less likely to encroach on the businesses provided by existing financial exchanges.
What many of us want from prediction markets bears more resemblance to the products of think-tanks than it does to any other institution that I’m aware of, and the basic ideas behind such uses of prediction markets are accepted by enough people that we can hope some wealthy donors would contribute to a charity that funds such markets. So it seems natural to me that we ought to focus a significant fraction of the thought about how to implement prediction markets into how to create a charitable organization for that purpose or persuade an existing think-tank to adopt that purpose.

Some of the examples are on a gray line between paternalism and what libertarians could promote as anti-fraud measures. For instance, lotteries are often marketed in ways that cause customers to misjudge the odds of winning, and requiring lotteries to tell customers how the odds compare to being hit by lightening would clarify the customers’ thinking a bit. The main problems with this idea are that it’s hard to implement due to the government’s bad incentives resulting from the money it gets from lotteries, and because it would be hard to write a law such that the lotteries couldn’t evade much of the effect by manipulating the number of prizes so that the chance of winning sounds better than whatever they’re required to compare to.

They have a clear argument that cocaine users have time-inconsistent preferences which indicate that they would be better off (according to the preferences that the users have in many contexts) if they could only buy cocaine with some delay (like the delay in getting prescription drugs) than if cocaine can be bought instantly. But it’s unclear what non-draconian enforcement methods would prevent them from buying quickly on the black market.

They have enough sense to ask whether there are sufficient private incentives to create paternalism that paternalistic laws are unnecessary. They reject this possibility for many cases because consumers aren’t aware of the biases that make paternalism a plausible option. But that leaves the puzzle of why voters would do a better job. Maybe they hope for undemocratic ways of accomplishing that, but I’m uncomfortable with arguments that advocate undemocratic government without being explicit that they are doing that, since ways of creating laws that aren’t overseen by voters tend to serve interests other than those of the voters.

They often have appropriate doubts about the wisdom of any particular instance of paternalism. For instance, they raise doubts about the benefits of food labeling laws by mentioning the Snackwell effect, where labels such as "reduced fat" cause consumers to overestimate how healthy a food is.

The degree of uncertainty over the wisdom of the kind of paternalism they talk about suggests to me that even if many people adopt the "asymmetric paternalism" approach to evaluating laws, a large fraction of proposed paternalistic laws will be unwise (due to a combination of uncertainty among experts and special interests misleading people). This leaves me suspecting that analyzing paternalistic laws on a case by case basis will still (at least until we get a political system the overcomes the biases that special interests exploit) result in an undesirable ratio of good to bad paternalistic laws, and that it is therefor better to use a general rule that such laws are bad (or assumed bad until proved good beyond some fairly demanding threshold). I think I see some bias by the authors toward underestimating the costs of disclosure laws. They say that it is a minimal burden on financial institutions to require them to provide customized disclosure to consumers that can be computerized. For the kind of large institution we often think of in this context, that seems reasonable. But what does that do to a small startup that is doing something novel? Even if the disclosure law is written well enough that it doesn’t restrict innovation (an unsafe assumption given that people at large institutions are often the only ones paying enough attention to influence how the disclosure laws are written, and would like to create barriers to entry), the programming work involved in a larger fraction of the total cost of running a small business than it is for a large institution. This has hard-to-measure effects on competition and innovation. I suspect the relative ease of measuring the benefits of these disclosure laws biases people toward ignoring these costs. For example, imagine a noncommercial open source software version of Prosper (a "Peer to Peer Lending Marketplace"). Is this possible with very minimal disclosure laws? I don’t know. Is it possible with complex disclosure laws? Probably not without big advances in AI.

They list three causes of status quo biases. I’m a bit surprised that they omit what my intuition tells me is one of the most important, namely that people care about their reputation for having made wise decisions, and sticking to whatever opinions they’ve formed in the past is more likely to promote such a reputation than a purely bayesian process would.

I have one minor complaint about their description of an experiment involving students who get lower grades if allowed to choose their own deadlines for handing in papers than do students who have evenly spaced deadlines imposed on them. The summary in this paper only says that students who choose deadlines unwisely have lower grades, and claims that demonstrates that deadlines cause benefits, even though that result might also have been caused by good students setting deadlines that look better (the paper which is the original source describing the experiment says that students given the choice do worse on average, so the experiment did a better job of demonstrating the value of deadlines than the summary indicates).

(I’ve updated the link to the one Hal found; the one I originally used seems to have stopped working.)