73 Comments

All wrong. In reality big firm, prestige lawyers often work very easy cases, and when they get a tough one, it's still an easy one because the sophisticated client doesn't expect much, hoping only to delay payment or for some kind of hail-mary like a credulous judge. Examples are easy to supply, just ask.

Expand full comment

Well, for one thing, if you rank lawyers based on their win percentages your 'top lawyers' will be the lawyers who found a way to end up with the easiest cases, and nothing more than that.

An idiotic idea threatens everyone's existing status, but that's not why the idea is not adopted; it is not adopted simply because it is idiotic.

Expand full comment

For several of the major professions, rating individuals by performance would be antithetical to the nature of the system. Law is a very clear example: justice is supposed to be fair for everyone. The goal of the system is that you don't get an outcome based on which lawyer you have, you get the outcome you deserve based on the facts and the law. To introduce a lawyer ranking system over the top of that would be wildly inappropriate - it would contribute to turning the justice system into nothing more than a sophisticated game.In medicine and education the situation is similar. The goal is not that there should be some good teachers, available to the rich or the lucky. The goal of the system is that every child should have access to decent teachers. To *systematically* rank and sort teachers by quality would be a grotesque admission that some kids just ain't gonna get the best. Medicine the same: I don't want to have to sort through the rankings. I want a competent, nearby, affordable doctor.I don't dispute that there is a significant element of class snobbery in opposition to rankings; but there are also very good reasons for not having them in many fields.

Expand full comment

It’s not hard to explain why top doctors, lawyers, etc. resist evaluation according to new metrics; by definition, the “top” lawyers/doctors are those that rank highly according to existing criteria, so the institution of new criteria constitutes a threat to their status. The employers of doctors/lawyers are themselves top doctors/lawyers, so it’s not especially surprising that there’s no push from employers. You point out that customers don’t seem interested in track records, but I think your observation is premature. It’s normal for a startup to have trouble recruiting customers even when its product is superior. Rule of thumb is you want to be 10x better than what already exists in the marketplace. Anything else means an uphill slog of marketing/“starting a movement” in the sense of https://web.archive.org/web... Note that there are examples of this sort of performance based evaluation taking over, but it doesn’t happen automatically; Moneyball is a good example (and there are fairly obvious reasons we should expect it to be slower in other fields, I think). So, short answer: humans are impressed by whichever attributes are currently statusful, and we are also unduly influenced by the opinions of statusful people, which results in the currently statusful attributes being self-reinforcing. This explains why e.g. we’re more willing to grade high school teachers according to strict quantitative criteria, since their profession is a less statusful one than that of the college professor.

Expand full comment

"Let me ask you this. There are (wouldn't you agree?) towering scientific geniuses. Why shouldn't there be towering business geniuses and legal geniuses?"

There have been very few "towering scientific geniuses" in the history of the world. Einstein may have been one (he was instrumental in multiple fields and had multiple big ideas, unlike most other famous scientists who were one-hit wonders), but overall competition is very though (even average scientists are very smart) and almost everything depends on being in the right field at the right time.

Are there Einsteins of business and law? Sure there have to be some, but these fields are even messier than science (Einstein could basically work out special relativity behind his desk at home, the math and physics either fit or they do not, if the world's best business or legal mind would be working as a patent office clerk there's no way he'd ever got to put that mind into practice). I certainly do not believe all those thousands of "top" businessmen and lawyers in the world have Einstein-caliber minds, but the whole point is that even if they are that smart and creative that really doesn't matter very much with all the random noise and viral effects in those fields.

I don't see what people find so genius about the Chewbacca defense. Confusion through overwhelming with nonsensical arguments has been known as a tactic since at least the ancient Greeks. Btw, has everyone forgotten that OJ Simpson lost a civil case about the murders and is currently serving a long sentence for other crimes? He only got a miracle one out of those three times even though he is a celebrity and each time had multiple experienced attorneys (say win probability of 0.65) on his payroll.

Expand full comment

The real estate thing is quite alien to me. Here in Europe we simply don't look for the "best" real estate agent. We simply see the agent as a menial middle man, with the house seling itself, perhaps this is because it's easier to look up government estimates of a house's value, government requirements to have an independent taxation before a sale and the selling prices of local houses being quite easily accessible public information.

Expand full comment

Robin just added real estate.

You're usually more loquacious...

Expand full comment

A popular celebrity who paid a team of the same number of "normal" lawyers some fee that they can live off of comfortably would have a near equal probability of getting their "miracle", though they'd miss the self-fulfilling prophecy effects that a "top" reputation can bring.

I think that's entirely false. (What are your reasons?) Would normal lawyers have come up with "if it doesn't fit, you can't convict"?

"Normal" lawyers are creatures of routine.

Let me ask you this. There are (wouldn't you agree?) towering scientific geniuses. Why shouldn't there be towering business geniuses and legal geniuses?

[Added.]I must reply to your elitist innuendo against the right to trial by a jury of your peers: Judges as far more attuned (as one might expect) to attorney prestige than are jurors.

Expand full comment

Sports, the business world, construction.

Expand full comment

A popular celebrity who paid a team of the same number of "normal" lawyers some fee that they can live off of comfortably would have a near equal probability of getting their "miracle", though they'd miss the self-fulfilling prophecy effects that a "top" reputation can bring. That last effect as well as the celebrity effect are excecarbated by judicial systems that rely heavily on laymen juries.

Expand full comment

I talked about outcomes not rankings. You can rank any of the kinds of cues that people use, not just outcomes.

Expand full comment

> they do believe in miracle workers (and of course the higher the fee, the greater or more frequent the miracles)

O.J. Simpson certainly got his miracle... There really is a big difference between what a millions-of-dollars legal defense team can do and a public defender can do.

Expand full comment

I'm not sure exactly what you mean by "school info", but if you mean which school they attended, then this is something that is known to applicants at the time they decide whether to make the investment. It affects the expected return, but not the risk.

Expand full comment

Typical rationalizations go like this

"He didn't go to a great school, but he really cares about me""He's really rude, but he went to Harvard"

These allow patients to rationalize their choices since there is sufficient vagueness in cause and outcome.

Relative rankings remove that fuzz factor and cause cognitive dissonance when you want to believe you're in good hands and/or have made a good choice. "He's in the bottom quartile of doctors in terms of outcomes, but he went to Harvard" doesn't have the same ring to it.

Expand full comment

There's some ancient research showing that the intellectual level of fellow students is the strongest predictor of gains.

Expand full comment

Yes school degrees "provide a rationale" but then so would outcome track records. This criteria doesn't distinguish between them.

Expand full comment