Why Academics Aren’t Bayesian

Bryan Caplan asks Why Aren’t Academic Economists Bayesians?:

Almost all economic models assume that human beings are Bayesians, … [but] academic economists are not Bayesians.  And they’re proud of it!

This is clearest for theorists.  Their epistemology is simple: Either something has been (a) proven with certainty, or (b) no one knows – and no intellectually respectable person will say more.  If no one has proven that Comparative Advantage still holds with imperfect competition, transportation costs, and indivisibilities, only an ignoramus would jump the gun and recommend free trade in a world with these characteristics. …

Empirical economists’ deviation from Bayesianism is more subtle.  Their epistemology is rooted in classical statistics.  The respectable researcher comes to the data an agnostic, and leaves believing “whatever the data say.”  When there’s no data that meets their standards, they mimic the theorists’ snobby agnosticism.  If you mention “common sense,” they’ll scoff.

I’ve argued that the main social function of academia is to let students, patrons, readers, etc. affiliate with credentialed-as-impressive minds.  If so, academic beliefs are secondary – the important thing is to clearly show respect to those who make impressive displays like theorems or difficult data analysis. And the obvious way for academics to use their beliefs to show respect for impressive folks is to have academic beliefs track the most impressive recent academic work.

So it won’t do to have beliefs bounce around with every little common sense thing anyone says, however informative those may be.  That would give too much respect to not-very-impressive sources of common sense. Instead, beliefs must stay fixed until an impressive enough theorem or data analysis comes along where beliefs should change out of respect for it.  Academics also avoid keeping beliefs pretty much the same when each new study hardly adds much evidence – that wouldn’t offer enough respect to the new display.

Relative to the Bayesians that academic economic theorists typically assume populate the world, real academics over-react or under-react to evidence, as needed to show respect for impressive academic displays. This helps assure the customers of academia that by affiliating with the most respected academics, they are affiliating with very impressive minds.

GD Star Rating
a WordPress rating system
Tagged as: , ,
Trackback URL:
  • Pingback: uberVU - social comments

  • http://www.hopeanon.typepad.com Hopefully Anonymous

    So this is an audience distortion problem? Does that mean that many academic economists are covering, that they’re internally bayesian but externally audience pleasing?

    Does the audience distortion problem disappear when bayesian-capable economists discuss economics behind closed doors?

    Is there a race to the bottom where economists that are behaving in a non-bayesian audience optimizing fashion attack economists that deviate from audience preference by discussing economics a more bayesian way? Or is a perfectly nontransparent coordination by bayesian-capable economists?

    What does your intuition say on this?

    • http://hanson.gmu.edu Robin Hanson

      Well when academics have substantial incentives to actually estimate accurately, and when that conflicts with celebrating impressive folks, academics would likely prefer to do their accurate estimation in private. It is not clear that strong conflicts arise much however.

  • http://www.stat.columbia.edu/~gelman/blog/ Andrew Gelman

    When youall say “Bayesian,” you’re referring to a model of rational behavior. When applied statisticians such as myself (or applied microeconomists, labor economists, etc.) say “Bayesian,” we’re talking about a particular set of statistical methods–which really have about the same claim to being rational as any other set of statistical methods do.

    To say this again: What you call Bayesian is not what I call Bayesian. And “Bayesian” is what I do for a living. This is not to say that you’re “wrong,” merely that it would make sense for you to be aware of other perspectives.

    See my bloggingheads conversation with your co-blogger Eliezer for more on this point, also I recommend you take a look at the Scholarpedia article on Bayesian statistics by David Spiegelhalter and Ken Rice.

    I use Bayesian methods myself–see my article/discussion/rejoinder in Bayesian Analysis on Objections to Bayesian Statistics for more on my motivations–but I certainly wouldn’t claim that people should be following a Bayesian approach because of any particular theoretical argument. Bayesian methods work for many problems I’ve worked on, that’s why I like them. But a method is only as good as its model. Historically there’ve been lots of Bayesians with silly models, and I think this has made some reasonable scientists wary (too wary, in fact) of Bayesian methods in general.

    • http://hanson.gmu.edu Robin Hanson

      I am indeed referring broadly to a model of rational belief, not more specifically a set of statistical techniques. Do you have a preferred alternative model economists should use for rational belief?

      • http://www.stat.columbia.edu/~gelman/blog/ Andrew Gelman

        I need more details to answer your question. Do you want a descriptive model of rational belief or a normative model of rational belief? If you want a model to do both, I think you’re asking for too much.

        For the descriptive model, I prefer Dave Krantz’s model of decision analysis based on goals and plans. (See also here.)

        For a normative model, I don’t really know. I would think that different people would have different normative models. For well-defined problems, I think classical utility theory has its strengths (see the chapter on decision analysis in Bayesian Data Analysis for some examples), but once you have to start patching it with things like “a taste for uncertainty,” “minimax regret,” and the like, I think it’s pretty hopeless.

        In any case, I think my above comment is relevant, because if an economist is described as Bayesian or not, this is a statement about the statistical methods that the economist uses, not a statement about their model for rational belief.

      • http://hanson.gmu.edu Robin Hanson

        From the point of view of this alternative decision theory are the patterns Bryan describes, of having no opinion until something is proven or of moving estimates all the way to each new empirical estimate ignoring the previous estimates and other data, rational? If not, that seems a distraction from the main point of this post.

      • http://www.hopeanon.typepad.com Hopefully Anonymous

        Professor Gelman,
        Although I think you offer great insights, I think you’re also being a bit parasitic in your posturing relative to Prof. Hanson.

        Prof. Hanson seems to me to be in good faith reaching for ways to optimize our social epistemology, whereas it seems to me that you’re sniping at the margins here, in a way “seems a distraction from the main point of this post”

        It feels to me like your policing is in the direction of reducing cognitive ambition, that I intuit is to our detriment.

    • Steve Rayhawk

      One translation of “a model of rational behavior” to “a particular set of statistical methods” might be, “Every economist should update their beliefs about economics after each new study in the samy way that a rolling cumulative Bayesian meta-analysis of all studies would update its conclusions.”

      But we have no (tractable) Bayesian model of all economic phenomena, and of the conditional probabilities of studies given economic phenomena, to use in that meta-analysis. And the Bayesian model of belief updating assumes logical omniscience, so an economist would have the same beliefs before reading a mathematical proof about comparative advantage as they would afterward.

      Another better translation might be, “Every economist should update their beliefs about economics, after each new study, to the average of the beliefs that they would predict from an imagined (tractable) rolling cumulative Bayesian meta-analysis of all studies, including the new one.”

      • http://hanson.gmu.edu Robin Hanson

        Don’t let the best be the enemy of the good; the standard would be to not knowingly deviate from what you expect a Bayesian would think.

      • http://www.stat.columbia.edu/~gelman/blog/ Andrew Gelman

        We have a saying in statistics: “The shitty is the enemy of the good.”

  • http://yudkowsky.net/ Eliezer Yudkowsky

    But surely all this is untrue of academics working in Artificial Intelligence. (Sorry, in-joke.)

  • http://www.rationalmechanisms.com richard silliker

    I would like to comment but am unable to because I lack a definition of: model of rational behavior. Can or would anybody be willing to help?

    • http://www.hopeanon.typepad.com Hopefully Anonymous

      Ha!
      I attempt one on my blog. I think agent persistence optimization is the model of rational behavior of an agent.

  • http://www.stat.columbia.edu/~gelman/blog/ Andrew Gelman

    I took a look at Caplan’s entry linked to above, and I can clarify that, as a practicing Bayesian statistician, I don’t think that statements of the form “Pr (hypothesis is true)” are very useful. (In Caplan’s example, he discussed “Pr (minimum wage causes unemployment).” I think you’re better off having a model of the effects of proposed policy changes (for example, an increase or a decrease in the minimum wage) and estimating the effects from there. I agree with Caplan that there can be a place for an economist’s prior distribution to enter here, but it can also make sense to try to understand what can be learned from a particular experiment. In that sense, I think things have gone the way Caplan would like. Despite the cited minimum wage study, and despite the overwhelming popularity in polls for raising the minimum wage by a lot, Congress has been reluctant to do much in that direction. Perhaps some of this reluctance can be attributed to the persuasive arguments of economic theory.

    Getting back to my earlier point, I think you should be very wary of going from the reasonable statement that people should reason logically to the highly flawed (in my opinion) statistical procedure of assigning probabilities to discrete hypotheses and trying to do Bayesian inference from there. If you want to do Bayesian inference, I think you’re better off building models, with all the costs and beneifts that has.

    And, no, in response to the other comment above, I am not trying to “posture” or “police.” As a blogger myself, I very much appreciate when people share their thoughts and comment on my blog, and I’m pretty sure that Robin feels the same way. Like “relativity” or “evolution,” he word “Bayesian” gets used in lot of ways and I think it can be helpful to understand its technical meanings.

    • http://hanson.gmu.edu Robin Hanson

      Seems to me Caplan is talking about an expected value: E[U|MW] – E[U|noMW]. I can’t see the point of a stat language that forbids discussing estimating such things.

      • magfrump

        Robin: the elimination of minimum wage would have social as well as directly economic consequences. These social consequences could give rise to different economic conditions.
        I might expect that E[U|MW] – E[U|noMW & Obama gives a speech to business owners] would be different from E[U|MW] – E[U|noMW & Obama gives a speech to the unemployed] or from E[U|MW] – E[U|noMW because they just stop enforcing it]

        In that way, having a specific policy change in mind gives us more reliable probability estimates.

        Also, I sincerely doubt that politicians’ decisions regarding minimum wages actually emerge from economic arguments (as opposed to: individual interest, political interest or sponsors, desire to appear strong/clever, pressure from party leaders). Richard, if you have some evidence of this, I would be interested to see it.

      • http://www.stat.columbia.edu/~gelman/blog/ Andrew Gelman

        Robin: I have no problem with expected values or utilities (at least in theory; in practice, there can be lots of difficulties in the implementation). But if you look at Caplan’s post that you link to, there’s nothing about E and nothing about U. It’s all about P. Here’s Caplan:

        The P(minimum wage causes unemployment|Card/Krueger study’s findings). Suppose P(CK findings|m.w. does cause unemployment)=.3, P(CK findings|m.w. does not cause unemployment)=.8, P(m.w. does cause unemployment)=.99, and P(m.w. does not cause unemployment)=.01. Then the conditional probability comes out to .3*.99/(.3*.99+.8*.01)=97.4%.

        I just don’t think this particular kind of reasoning makes sense or is a helpful way to do statistics. The world (at least, the social science world) just isn’t discrete in this way.

        Anyway, you’re free to disagree with this–many statisticians do. I just wanted to isolate the way in which what Caplan (and many others) think of casually as rational Bayesiansm is not something that fits in well with my conception of Bayesian statistics.

  • http://www.weidai.com Wei Dai

    Robin, if you believe that academic belief is largely about signaling rather than truth, why do you often cite academic consensus to support your own positions, without adding a disclaimer like “but of course academic belief is mostly about signaling”? (See here and here for a couple of recent examples.) Do you expect that most of your readers are already familiar enough with signaling theory and how it applies to academia, that they can take this into account and not over-react to the evidence of academic consensus that you provide, even without an explicit disclaimer?

    • http://hanson.gmu.edu Robin Hanson

      Most expressed human opinions are largely about signaling rather than truth. If we had a way to find what folks with expert level knowledge thought that focused mainly on a truth motivation, we’d love to defer to that. But absent prediction markets, that seems hard to find.

      • http://www.weidai.com Wei Dai

        It does seem that academic beliefs tracks truth better than most expressed human opinions, even though they’re both about signaling.

        Markets (ordinary ones) harness rent-seeking to improve social welfare. Academia seems to harness status-seeking to improve knowledge, but perhaps in a way that we don’t understand (certainly not as well as we understand markets). Prediction markets would harness rent-seeking to improve knowledge, but interest in such institutions seem too low to make them a viable alternative to academia.

        Is anyone working on trying to better understand academia and perhaps improving it on the margins, while leaving its signaling function intact?

      • http://yudkowsky.net/ Eliezer Yudkowsky

        “Rent-seeking” has a particular meaning in economics. Is there a special economic term for ordinary-desire-for-money or do we just say “greed”?

      • http://www.hopeanon.typepad.com Hopefully Anonymous

        Wei, I disagree that academia primarily harnesses status-seeking to produce knowledge. I think rather that academia is more of a coordination among knowledge seekers to optimize their knowledge-seeking. In other words, I think the credentialing of third party non-academics, etc. is to fund the knowledge-seeking, with some drift towards status driven behavior, like graduation ceremony pomp and circumstance. Faust is the academic archetype, it seems to me, like Midas is for the capitalist and Narcissus for the artist. Academia isn’t the efficient route to status. I think status-games represent a corruption of the academic, or a concession to the non-academic world.

      • anon

        Eliezer, I suspect Wei Dai meant “profit-seeking”, not rent seeking. Generally speaking, rent-seeking behavior does not increase social welfare.

  • http://cob.jmu.edu/rosserjb Barkley Rosser

    Much of this is simply due to entrenched models of teaching. So, pretty much all intro econometrics courses assume classical frequentism, and the standard BLUE theorems for such workhorse methods as OLS all assume a CF framework. It is only at a much more advanced and specialized level that one is likely to encounter Bayesian methods, particularly in time-series analysis. This is in contrast to medical research, where I understand that Bayesian methods have come close to beating out the CF ones.

    Of course, many economists when confronted with the more general arguments will grant that Bayesianism has a lot of appeal in principle. But as they only known CF models, they kind of let it go, sort of a “we are all Bayesians now, but only in theory for most of us” view.

  • http://bizop.ca michael webster

    Robin, I thought that your signalling theory was designed to explain why rankings of prestigious institutions doesn’t correlate well with what independent rankings we have of the ability of individuals in those institutions. Am I wrong?

  • Pingback: Why decent philosophers use too much jargon : The Uncredible Hallq

  • Pingback: <corruptmemory> » blog » The futility of “direct” religious discussion

  • Pingback: Alexander Kruel · A Guide to Bayes’ Theorem – A few links

  • Pingback: Letters to Doubting Thomas (a review) : The Uncredible Hallq