How Much Defer To Experts?

Bryan Caplan argues the public has very poorly informed political beliefs, best described as "irrational," and his main evidence is that they are strong and yet differ greatly and systematically from beliefs of professional economists, after correcting for ideology, income, and other characteristics.  Dan Klein and Tyler Cowen complain that, compared to most economists, Bryan’s beliefs unusually far from median economists’ beliefs.  Tyler

Dan’s key point is that you, in fact, differ radically from the professional consensus (as Arnold writes as well) and that the argument is self-undercutting if you simultaneously erect expert consensus as a relevant benchmark. I don’t see that you have yet replied to this.

Bryan replies

I only embrace a presumption of expert competence; it’s where I start, not necessarily where I finish.  Still, Tyler is entirely correct to say that I’m not an average economist. How do I account for the difference? In large part, I think that other economists have failed to fully free themselves from anti-market bias; … the best I can do is argue issue-by-issue, and see if I can make a dent. … Another large chunk comes down to different values. … I’m a staunch libertarian. I’m extremely meritocratic. … I look down on patriotism and piety of every kind.

It seems to me that Bryan’s intuition is that a large disagreement with experts is a strong positive sign of irrationality, but that a small disagreement is only a weak sign about rationality.  Even if Bryan is an unusual economist, he still mostly agrees with them, making his overall disagreement "small."  But this raises the obvious question:  how much disagreement with experts is enough for a strong sign of irrationality? 

GD Star Rating
loading...
Tagged as:
Trackback URL:
  • http://www.jame5.com Stefan Pernar

    How much disagreement with experts is enough for a strong sign of irrationality?

    At the time of Einstein publishing his theory of relativity he disagreed fundamentally with the majority of experts – was he being irrational? It seems that disagreement in itself is not a sign for irrationality rather that for being irrational one would have to disagree qualitative not quantitatively.

  • Caledonian

    The degree to which your position differs from others’ is irrelevant. The only thing that matters is how well your own position can be defended.

    If the experts are irrational, why would rejecting their arguments be a sign of irrationality?

  • outeast

    At least some disagreement with economists seems to me to be rooted not in a rejection of their de facto expertise but in differing moral commitments, and this need not be irrational; non-rational, maybe, in that moral commitments are not generally rationally derived, but not irrational. This is especially evident in instances where economists have to calculate the value of things such as human life, animal species loss, and so on.

    I think it is the grounds for and nature of the disagreement, rather than the extent thereof, that is the better indicator of irrationality.

  • http://profile.typekey.com/robinhanson/ Robin Hanson

    Stephan, we have many other signs about Einstein besides this one fact of his disagreement.

    Calend, surely the fact that others disagree is relevant information.

    Outeast, disagreements about morality are also info about which morality is right.

  • outeast

    Robin, that is a truism but I’m not sure it’s very helpful. The sense in which moral commitments are ‘right’ is not the same as that in which a given theory is ‘right’.

    Rejection of a theory which rests on expert knowledge – say, of the theory of evolution – is irrational: the data which goes into the description on evolution requires no subjective value judgements. In contrast, rejection of a cost-benefit analysis (say) which assigns a particular monetary value to the loss of a species may not be irrational despite rejecting an expert premiss: the principle that a species has a very high intrinsic value regardless of any input into economic processes is a moral commitment which cannot be proved ‘right’ or ‘wrong’ in a factual sense.

    For that matter, I’d argue that a commitment to the kind of utilitarianism that leads to a choice of torture over specks is also a moral commitment, not a rational one; as of course is the rejection thereof.

  • michael vassar

    Along with “how should we identify experts” this is pretty much the central recurring question of this blog, is it not?

    It seems to me that “start with a probability distribution roughly matching the belief distribution of the expert community but slightly broader and then update normally rather than double counting” is a reasonable starting place. To some degree one also has to force coherence among beliefs. This doesn’t seem like a new insight, just a description of what aspiring rationalists do almost by definition if they don’t get severely confused and cease to be aspiring rationalists.

  • http://profile.typekey.com/sentience/ Eliezer Yudkowsky

    I think Bryan’s point is that he is an expert. Novices should agree with experts and experts should “make up their own minds” but without any bias toward disagreement, based on all the experimental evidence they have time to read about.

    An oversimplification but with a grain of truth. My father told me never to argue with physicists about physics, but I’m sure he never meant that injunction to apply to other physicists. (I haven’t always obeyed that injunction, because sometimes some physicists say silly things about falsifiability or Occam’s Razor, not realizing that these are quantitative concepts, but that’s a whole separate topic.)

  • http://profile.typekey.com/robinhanson/ Robin Hanson

    Outeast, you seem to claim that one cannot be expert on morality. If so, how sure are you that one can know anything? If one can learn anything, it would seem one could learn more, and become expert.

    Michael, I’m not sure what you mean by “start with.” Info about expert opinion is spread across time just as all other info is.

    Eliezer, while novices should pay more attention to expert opinion, it also seems informationally relevant for experts as well.

  • outeast

    Robin: OK, I think the ‘expert on morality’ thing is a bit of a distraction. My point was more meant to be that moral commitments frequently constitute premisses in particular expert theories, models, and calculations, and it is possible to reject expert opinion on the grounds of disagreement with such moral assumptions without this being irrational*.

    As a concrete example: the economist Bjoern Lomborg has done a cost-benefit analysis of global warming. A non-economist would be irrational to reject his work as bad economics because he or she would not have the expert knowledge needed to analyze this aspect of his work; but he or she could perfectly legitimately (and not irrationally) reject it on the grounds that his valuations of species loss (for example) are grossly in error. This would be the result of a moral commitment to the value of species diversity which would not be irrational even though it might not be based on a rational calulation of value.

    *With ‘irrational’ denoting a negative value of rationality. Flying in the face of the evidence is irrational; a value judgement (‘I like cheese more than chocolate’) is neither rational nor ir-. Similarly with moral commitments… though these may of course be rationalized:)

  • http://dualanalog.com Matt

    There is a lot of generalizing happening when you label a group of people experts and decide that there is one point of view speaks for all of them. There will be deviations of expertise (Is a grad student considered an expert? What about a journalist who has covered economics for many years?) as well as those experts opinions on the matter.

    I think Bryan’s general point stands though. The more expertise someone has in a field, the more likely their opinions will be based on a rational assessment of available facts. There will still be deviations from the norm, as biases still exist and facts can be interpreted differently. However, those biases are likely to play less of a role than in someone who has little experience in the field and are highly influenced by bias and have little experience or knowledge to combat said violence.

    The fact that Brian, or any other economist, has a view that differs from the norm, does not, by itself, disprove his original hypothesis.

  • http://entitledtoanopinion.wordpress.com TGGP

    I think we’ve discussed expertise in morality before. I make the meta-ethical assertion that the set of moral facts is empty, hence there are no experts. There may be historians of moral thought who know a lot about what this or that person has said, but there is no one that knows any more than anyone else within the subject of morality, just as there cats cannot see better than any other animal in absolute darkness.

    Economists may have different values than the general public on average, but Caplan uses an “enlightened public” and ideological controls to test how important the effect is, and knowledge of economics/education/IQ still seem to be the biggest factors. Much of the disagreement was over strictly positive matters, and a great many members of the public stated that they would favor different policies if the positive facts were different.

  • http://profile.typekey.com/sentience/ Eliezer Yudkowsky

    While there may be a defensible position that experts should pay attention to experts, I think it has to be defended over and above the position that novices should pay attention to experts.

  • Bill Abbott

    It seems to me that if Bryan’s claim were that the professional consensus was the standard of rationality, then the criticism would have more strength. If the claim is that professional consensus simply indicates the direction of bias, then the benchmark of rationality may be beyond the professional consensus.
    So basically, direction of disagreement with experts matters. If there is a bias present, then disagreement from experts in the direction of the public will tend to be a stronger sign of irrationality than a similar degree of disagreement in the direction away from the public belief.
    Which I guess is Bryan’s point? The experts are not unbiased, they are just less biased, so the rational position is a “small” distance from expert consensus in the direction away from the public.
    How much disagreement is enough? Tough question, seems like the strength and nature of the biases involved would matter quite a bit, and would vary a lot amongst the issues.

  • michael vassar

    Robin: I mean use expert opinion as a prior, but I’m not sure that we agree about how to think about priors.

  • http://www.defendingthekingdom.com/ Ian

    Someone who understands both positions and can explain why one of them is wrong and the other right is more likely to be right in any given argument.

    Most people cannot explain economists’ positions on such things as free trade or the minimum wage; they know only that they distrust the first and like the second. Economists have good explanations for why lay people are misguided and why it is the economists who are correct on these topics.

    Most creationists do not understand and cannot explain the theory of evolution. They show their ignorance every time they express incredulity that some animal part or another could arise by “random chance”. Most scientists can explain exactly why creationism is inane and why evolution is the superior theory.

  • Caledonian

    My father told me never to argue with physicists about physics

    Did he also tell you never to question their claims? And how exactly do you determine who counts as a ‘physicist’ that you shouldn’t argue with?

  • Douglas Knight

    michael vassar,
    What is the role of your second paragraph? It sounds like an elaboration of the two “central questions,” but you also say “start with” which makes it sound like what to do without answering them.

  • Stuart Armstrong

    how much disagreement with experts is enough for a strong sign of irrationality?

    I think it depends of the broadness of expert opinion. If all of expert opinion is clustered around a narrow claim – that the mass of the electron is 9.109 382 15(45) × 10^(–31) kg, for instance – then it is highly irrational to be far off the consensus without exceptionally strong supporting evidence.

    But if expert opinion is broad (pick practically any social science), then one can be further away from this consensus without being neccessarily irrational. To steryotype, if you are more than two standard deviations away from expert opinion, you’re probably irrational (unless the distribution of expert opinion has several peaks or is otherwise deranged).

    But is Bryan mostly agreeing with other economists in the sense that all his opinions are close to the consensus (in which case, no problem), or just that he shares most of their opinions but vehemently disagrees with some (in which case, he is probably being irrational)? There’s also the issue of whether his disagreements are in a systematic political direction; that increases the chance of him being irrational.

    Two comments on this “no more than two standard deviations” idea:
    1) Einstein was very close to the consensus of expert opinion. His theory was utterly bizzare, but the predictions of his theory were close to those of the consensus theories, in most cases that had been tested at the time. Closeness should be judged in predictions, not the justication of those predictions.
    2) This set-up has a perverse incentive – those who try and push a wrong-headed idea will do all they can to broaden the expert consensus, or at least give the appearance that it is broad. Cue the tactics of Creationists, etc…

  • Shakespeare’s Fool

    When Muhammad Yunus disagreed with the banking
    experts about the creditworthiness of poor Bangladeshi
    women, Yunus was right. His disagreement was total.
    And he was totally right.

    When Bohr and Einstein debated quantum uncertainty,
    did the average physicist get to choose a side?

    Even now, nearly 30 years after Wilson and Penzias
    received the Nobel for their work on cosmic
    microwave background radiation, there are physicists
    and many others who have not discarded their belief in
    universal increasing entropy. If a 6th grader knows the
    physicist is wrong, shouldn’t she disagree with him?

    If an investor disagrees with a professor of economics
    or finance who expounds the efficient market theory,
    who is the expert?

    If a gynecologist advises her patients to use or not use
    hormone replacement therapy, she must disagree with
    experts whatever her advice.

    In “How Doctors Think,” Jerome Groopman gives
    but a few of the instances when patients should
    disagree with their physicians.

    Was not Agamemnon wrong not to have disagreed
    with the priest about the disposition of Iphigenia?

    John

  • http://profile.typekey.com/robinhanson/ Robin Hanson

    Shakespeare, if I gave you a long list of lottery winners would you buy lottery tickets?

  • Nick Tarleton

    Even now, nearly 30 years after Wilson and Penzias
    received the Nobel for their work on cosmic
    microwave background radiation, there are physicists
    and many others who have not discarded their belief in
    universal increasing entropy. If a 6th grader knows the
    physicist is wrong, shouldn’t she disagree with him?

    Background radiation means entropy isn’t increasing? Am I missing something obvious?

  • Caledonian

    Calend, surely the fact that others disagree is relevant information.

    No. Only the arguments and the evidence supporting them are relevant. Crowds may occasionally have wisdom, but too many people mistake mobs for crowds for the principle to have general utility. I have never encountered an issue so resoundingly resolved by the available data that some fool couldn’t disagree on the matter.

  • Nick Tarleton

    In principle, if someone is even a tiny bit more likely to hold true than false beliefs, their beliefs are evidence.

    In practice, it’s probably useful to disregard mob disagreement, but it’s silly not to take expert disagreement into account. This doesn’t mean you have to agree with experts, just that disagreeing with them involves taking on an additional burden.

  • Shakespeare’s Fool

    Robin,

    Interesting that you agree with all my points but not
    with what I might surmise you think my
    generalization is. (Or did you just not list your
    disagreements?)

    Charles Mackay wrote the book “Extraordinary
    Popular Delusions and the Madness of Crowds” If
    my IQ were as much higher than mine as your
    accomplishments lead me to believe yours is, I would
    be tempted to write “Extraordinary Academic
    Delusions and the Madness of Experts.” Even
    Einstein would have an entry. And, because it has
    always been thus, the book would have many errors
    of its own. And all-too-likely a madness or delusion
    or two.

    I am not saying I think experts are always wrong,
    just often enough and sometimes obviously enough
    that we will all find times when total disagreement is
    totally justified (even though we won’t always be
    right – even though some of us will never be right.)

    John

  • Shakespeare’s Fool

    Nick Tarleton,

    Yes. You are missing something as blindingly
    obvious as the difference in conditions between the
    big bang and the present. Easy to miss because
    experts have been telling you the opposite of what
    Wilson and Penzias made obvious.

    John

  • Nick Tarleton

    The question of why the initial conditions of the universe had very low entropy is an acknowledged open problem, if that’s what you’re talking about. Your intentional vagueness gives me little confidence in the strength of your position.

  • Roy Haddad

    The opinion of a large body of people and that of an individual should be treated very differently. Those of a large body of people are unlikely to follow from a consistent logical position. In fact, the rigorous application of logic tends to result in a higher variability in opinion, which will tend to cancel out, compared to a common bias – the Miracle of Aggregation’s evil twin operates opposite the desirable direction, stripping out the logic, leaving more bias.

    Another angle: it is very unlikely that for some given issue that the experts are wrong and the general population is right because it is unlikely that the general population would have a specific insight that the experts are either unaware of or unable or unwilling to incorporate, while it would be more credible that the experts are wrong and a particular individual is right because of some specific insight.

    In a nutshell: disagreement with experts is strong evidence against crowds, but not so much against specific persons.

  • http://www.mnuez.blogspot.com mnuez

    Yes, Bryan Caplan IS a cold-blooded asshole

    Back when Bryan “LuckyGenes” Caplan was all the rage for his ivory tower giggling at the masses of Americans who “are irrational” about economics owing to their silly empathies such as those that would spare their non-productive grandparents the ice-floe, I was one of the few who called him out on it.

    I pointed out that Caplan’s book had nothing to do with people being misinformed about economics and everything to do with their wanting to eat – even if allowing them to starve WOULD produce really cost-effective fertilizer for the wheat-fields. My cries regarding this emperor’s non-existent raiments were considered ludicrous or silly. “Caplan is no ideologue”, they said. “He’s simply an honest-broker academic!”

    Fools.

    Caplan writes non-hysterically about being “rational”, “factual”, non-xenophobic and all sorts of other high-standing qualities and the masses of pseudo-intellectuals trample all over each other in their attempts to jump aboard his “We’re Smart!” Express.

    Well, some of Caplan’s fellow economists have called him on his charade of claiming to represent the views of economists at large and he’s come out with a few sentences which may shed some light on the policies that he advocates. They may or may not be YOUR policies, but they’re hardly in line with the moral tastes of your average academic, American or human being.

    From the jackass’s mouth:

    I am also well outside the mainstream of American values. Consequences aside, I’m a staunch libertarian. I’m extremely meritocratic. I disagree with the aims, not just the methods, of egalitarianism. I look down on patriotism and piety of every kind. The list goes on.

    So suppose my book led the world to reject anti-market, anti-foreign, make-work, and pessimistic bias in all their forms, with all their empty promises? How much would policy change as a result? My best guess is that policy would be a little less libertarian than Milton Friedman would have wanted. We’d still have substantial redistribution to relatively poor Americans and the American elderly – much of it funded by surtaxes on guest workers. We’d see educational vouchers, not the separation of school and state (my first choice); and those vouchers might not even be means-tested. In short, we’d have big changes, but I’d still have plenty to complain about.

    That’s me.

    Yes, that IS him. That’s EXACTLY him.

    My original articles on this blog regarding Caplan and his fellow travelers are here. They’re worth reading:

    The Economist and You

    The Conversation Continues

    mnuez
    http://www.mnuez.blogspot.com

  • http://profile.typekey.com/robinhanson/ Robin Hanson

    Roy, every member of the crowd is a specific person.

    I unpublished an insulting argument-free comment by mnuez.

  • Shakespeare’s Fool

    Nick Tarleton,

    The condition of the universe after the big bang is
    indeed a problem for the theory of increasing entropy.
    Since that early state of high entropy, what we can
    observe of the universe has declined in entropy. There
    is no evidence of which I am aware that entropy is not
    continuing to decline. If you have such evidence,
    please let me know what it is.

    John

  • Caledonian

    In principle, if someone is even a tiny bit more likely to hold true than false beliefs, their beliefs are evidence.

    If the beliefs of people who are slightly more likely to be correct than wrong count as evidence, then I can control what conclusions you reach simply by getting a sufficiently large number of people to reach consensus. Any required amount of ‘evidence’ can be constructed at will.

    In reality, we recognize that some kinds of evidence are more primary than others, and there’s a certain hierarchy at work. Primary evidence trumps any amount of secondary, and secondary trumps any amount of tertiary.

    Who are you going to believe? The experts? Or your lying eyes?

  • Caledonian

    Um… the universe has been increasing in disorder, which in informal speech is described as an *increase* in entropy, not a decrease.

  • Nick Tarleton

    If the beliefs of people who are slightly more likely to be correct than wrong count as evidence, then I can control what conclusions you reach simply by getting a sufficiently large number of people to reach consensus. Any required amount of ‘evidence’ can be constructed at will.

    I didn’t say it couldn’t be a really, really tiny amount of evidence. Even if you could get five billion people to agree on something, it could still be weaker than a strong expert opinion. Besides, if I know you’re engaging in mass deception, I may decide that most people’s opinions actually are unrelated to the truth.

    In reality, we recognize that some kinds of evidence are more primary than others, and there’s a certain hierarchy at work. Primary evidence trumps any amount of secondary, and secondary trumps any amount of tertiary.

    Read:
    http://yudkowsky.net/bayes/bayes.html
    http://yudkowsky.net/bayes/technical.html

  • Caledonian

    Even if you could get five billion people to agree on something, it could still be weaker than a strong expert opinion.

    Five billion strong ‘expert’ opinions are weaker than a single piece of direct data. It’s the nature of opinion.

    What makes the experts’ judgements valuable is the data underlying them. If the data leads to their conclusions, their judgements are valuable. If not, they’re not. Beliefs are – or should be – reflections of the evidence. They are not evidence in themselves.

    Yes, yes, I’ve read it all before.

  • Roy Haddad

    every member of the crowd is a specific person.

    So?

  • Shakespeare’s Fool

    Caledonian,

    As I asked of Nick Tarleton I ask of you:
    If you have evidence that the universe is
    increasing in disorder, please let us know
    what the evidence is.

    John

  • Caledonian

    As I asked of Nick Tarleton I ask of you:
    If you have evidence that the universe is
    increasing in disorder, please let us know
    what the evidence is.

    Um… basic thermodynamics? Logic? Hubble’s Constant?

  • http://profile.typekey.com/robinhanson/ Robin Hanson

    Shakespeare, increasing entropy is an extremely standard well-established physics result. This is not to claim it could not possibly be in error, but surely this puts the burden on you to at least offer an argument why that consensus is mistaken. And the comments of this post is not exactly the right forum for presenting such a dramatic result.

  • Shakespeare’s Fool

    Robin Hanson says: “Shakespeare, increasing entropy
    is an extremely standard well-established physics
    result.”

    Robin,
    As is so often the case, you are right.

    Which is why I thought it might be appropriate for a
    discussion of disagreements with experts.

    John

    P.S. Perhaps an appropriate forum would be my ad hoc
    hotmail.com e-mail account EntropicConundrums .
    See you there?