Beware Prestige-Based Discretion

Before the modern world, most jobs had a big physical component. And so physical ability (strength, speed, stamina, coordination, etc.) was one of the main things people tried to show off. Yes, people did try to show off physical abilities on the job. But when people got serious about showing off, they created special off-the-job contests, such as races and games.

These special contests made it much easier for observers to see small ability differences. For example, you might watch messengers all day on the job running from place to place, and though you’d get a vague idea of which ones were faster, you couldn’t see fine differences very well. But a race controls for other variation by having contestants all start at the same time on a line, and all run straight to a finish line. So even if one runner beats another by only a fraction of a second, observers can still see the difference. Other kinds of special contests also reduce noise, making it easier to see smaller ability differences.

When people can choose between competition forums with more and less noise, signaling incentives will induce them to choose forums with less noise. After all, competitors who choose forums with more noise will be seen as trying to hide their lower abilities among the noise.

So if messengers who wanted to show off their running abilities had a lot of discretion about how messenger jobs were arranged, they’d try to make their jobs look a lot like races. Which would help them show off, but would be less effective at getting messages delivered. Which is why people who hire messengers need to pay attention to how fast messages get delivered, and not just to hiring the fastest runners. Just hiring the fastest runners and letting them decide how messages get delivered is a recipe for waste.

In the rest of society, however, we often both try to hire people who seem to show off the highest related abilities, and we let those most prestigious people have a lot of discretion in how the job is structured. For example, we let the most prestigious doctors tell us how medicine should be run, the most prestigious lawyers tells us how law should be run, the most prestigious finance professionals tell us how the financial system should work, and the most prestigious academics tell us how to run schools and research.

This can go very wrong! Imagine that we wanted research progress, and that we let the most prestigious researchers pick research topics and methods. To show off their abilities, they may pick topics and methods that most reduce the noise in estimating abilities. For example, they may pick mathematical methods, and topics that are well suited to such methods. And many of them may crowd around the same few topics, like runners at a race. These choices would succeed in helping the most able researchers to show that they are in fact the most able. But the actual research that results might not be very useful at producing research progress.

Of course if we don’t really care about research progress, or students learning, or medical effectiveness, etc., if what we mainly care about is just affiliating with the most impressive folks, well then all this isn’t much of a problem. But if we do care about these things, then unthinkingly presuming that the most prestigious people are the best to tell us how to do things, that can go very very wrong.

GD Star Rating
loading...
Tagged as: ,
Trackback URL:
  • FuturePundit

    We need great metrics to measure performance. Sometimes (often in fact) we need to put a lot of work into developing and testing metrics for relevance and accuracy. Too often the great metrics do not get developed and poor metrics create a misleading picture.

    We run into the very big problem that things that get measured end up skewing incentives toward optimizing those measures at the expensive of other aspects of the system not so easily measured. This has led many companies down.

    Ditto academic institutions. Look at the journals that exist to give academics places to publish. It is much easier to measure quantity than quality when every product is unique.

    Metrics are hard.

    • http://overcomingbias.com RobinHanson

      But we often have plenty good metrics that people don’t want to use.

      • David Condon

        I believe you are referring to algorithm aversion with that statement.

      • FuturePundit

        Agreed. In some kinds of institutions the biggest obstacle is an unwillingness to use metrics. When is it a sign of unfortunate incentives and when it is a sign of innumeracy and failure to appreciate the possibilities?

      • http://overcomingbias.com RobinHanson

        I think the problem is more fundamental; basic human prestige programming.

  • Lord

    It could be worse though. We could let non experts judge. The problem is non experts are even in less position to judge. They won’t know the problems or possible solutions, they won’t know what to look at, what to measure, how to measure it, or whether it is indeed measured. The best one can hope for is multiple independent groups of experts with high ethics cross checking each other but able to pursue their own path.

    • David Condon

      I strongly agree with this. The alternative to experts is often laymen who often have neither the incentive nor the ability to discriminate between better and worse options. I think this is an argument for less expert involvement, but I don’t think it’s an argument for greater populist involvement. The danger of inadvertently implying support for greater populism is such that I think whenever making an argument against experts, it is very important to stress that an argument against experts is not an argument in favor of populism.

      • http://juridicalcoherence.blogspot.com/ Stephen Diamond

        One must also look at the balance of influences. Even under the reign of experts, there are countervailing pressures to mediocrity.

  • free_agent

    It seems like this problem is intensified (1) if the public doesn’t have a good way to evaluate the properties it really wants, or (2) if the highest-prestige workers have extraordinarily higher productivity (and so are in a position to demand the work be structured in the way they want as part of their pay).

  • Sharknado

    I agree with you, and I also think your warning touches on a very specific case of a general problem: we tend to award discretion based on demonstrated levels of technical skill, rather than based on demonstrated levels of altruistic virtue. That works fine when the discretion is about a technical topic, e.g., how many milligrams of this drug should be prescribed — but it works terribly when the discretion is about a social topic, e.g., who should have access to this drug.

    I’m not sure how likely professionals are to abuse their discretion *specifically* for the purpose of showing off their high skill level, but I’m sure professionals abuse their discretion all the time for other reasons, because we barely have any controls designed to ensure that professionals who have the most discretion also have the highest ethical standards.

  • Viliam

    If there are many prestigious people, they will be tempted to choose the part of the field where it is easier *in general* to show off. But if there are only a few prestigious people (in extreme case: only one), they may choose the part of the field where they *personally* have an advantage.

    For example, a senior developer in an IT company may insist that everyone must use the technology the senior developer has most experience with, even when it does not fit the project’s needs. (Even if it is *not* the technology that developers *in general* would use to show off.) Of course, if other developers will suggest another technology, it is easy to accuse them of doing exactly this.

  • arch1

    It *is* famously difficult fto get health professionals to hand-wash diligently, though that is critical to their ostensible mission. I gather the same is true of checklist use. And come to think of it, the perverse, pervasive, persistent tradition of abysmally-scrawled prescriptions affords writer *and* reader a minor chance to show off with an apparently acceptable level of collateral injuries and deaths.

    Do such things correlate with the individual health professional’s level of prestige? Inversely with the patient’s degree of connection to the professional?

    PS. That said it’s not all about prestige. I’ve got to believe that lots of these folks try harder to do a good job than necessary to maximize their prestige. And other extraneous factors can have big impacts too: Recall the 2011 peer-reviewed study of 1,000 Israeli
    parole applicants which found that “You are anywhere between two and six times as
    likely to be released if you’re one of the first three prisoners considered
    [during a break-less session] versus the last three prisoners considered.”

    • http://juridicalcoherence.blogspot.com/ Stephen Diamond

      The study you cite and its offspring is at the center of the “replicability crisis” in social psychology. ( http://www.slate.com/articles/health_and_science/cover_story/2016/03/ego_depletion_an_influential_theory_in_psychology_may_have_just_been_debunked.html )

      Concidentally, this crisis may be an excellent example of Robin’s topic. There is little prestige in replicating the findings of others. All the fun (read signaling value) is in designing clever experiments, not redoing those designed by others.

      • arch1

        Thanks Stephen. Aware of the replicability issue generally (and how enticing a meme the study had produced) I did a cursory search for followups found only noncritical summaries. I *was* impressed that the article was edited by Daniel Kahneman (poke at myself there:-)

        The Slate article is fascinating but doesn’t seem to mention the parole study. Prompted by your comment I did find a critical response to the parole study by Weinshall-Margell and Shapard, and a response to that by the original study authors (Danzinger et al), both in 2011. But I gather from your comment that there’s more than just that. Do you have any pointers?

      • http://juridicalcoherence.blogspot.com/ Stephen Diamond

        To my knowledge, no one has attempted to replicate the parole study. But the theoretical basis for interpreting it is thrown into question by the failure of replication attempts of studies supporting the ego-depletion/decision-fatigue concept.

  • https://plus.google.com/+AlexeyFeldgendler Alexey Feldgendler

    This might be one of the reasons why so many software developers prefer to write “interesting” tools themselves, especially compilers, instead of reusing them. “I wrote a compiler” is such a great claim to make, even if the compiler was written (and a programming language invented) as part of something like developing an accounting app.

  • Pingback: Overcoming Bias : Against Prestige

  • Pingback: Cornerstone Draft | Mark Finnern

  • Pingback: Prestige: Proxy to Select & Manage Research Projects | Delightful & Distinctive COLRS