Bryan Caplan asks Why Aren’t Academic Economists Bayesians?: Almost all economic models assume that human beings are Bayesians, … [but] academic economists are not Bayesians. And they’re proud of it!
Robin, I thought that your signalling theory was designed to explain why rankings of prestigious institutions doesn't correlate well with what independent rankings we have of the ability of individuals in those institutions. Am I wrong?
Wei, I disagree that academia primarily harnesses status-seeking to produce knowledge. I think rather that academia is more of a coordination among knowledge seekers to optimize their knowledge-seeking. In other words, I think the credentialing of third party non-academics, etc. is to fund the knowledge-seeking, with some drift towards status driven behavior, like graduation ceremony pomp and circumstance. Faust is the academic archetype, it seems to me, like Midas is for the capitalist and Narcissus for the artist. Academia isn't the efficient route to status. I think status-games represent a corruption of the academic, or a concession to the non-academic world.
Much of this is simply due to entrenched models of teaching. So, pretty much all intro econometrics courses assume classical frequentism, and the standard BLUE theorems for such workhorse methods as OLS all assume a CF framework. It is only at a much more advanced and specialized level that one is likely to encounter Bayesian methods, particularly in time-series analysis. This is in contrast to medical research, where I understand that Bayesian methods have come close to beating out the CF ones.
Of course, many economists when confronted with the more general arguments will grant that Bayesianism has a lot of appeal in principle. But as they only known CF models, they kind of let it go, sort of a "we are all Bayesians now, but only in theory for most of us" view.
It does seem that academic beliefs tracks truth better than most expressed human opinions, even though they're both about signaling.
Markets (ordinary ones) harness rent-seeking to improve social welfare. Academia seems to harness status-seeking to improve knowledge, but perhaps in a way that we don't understand (certainly not as well as we understand markets). Prediction markets would harness rent-seeking to improve knowledge, but interest in such institutions seem too low to make them a viable alternative to academia.
Is anyone working on trying to better understand academia and perhaps improving it on the margins, while leaving its signaling function intact?
Most expressed human opinions are largely about signaling rather than truth. If we had a way to find what folks with expert level knowledge thought that focused mainly on a truth motivation, we'd love to defer to that. But absent prediction markets, that seems hard to find.
Robin, if you believe that academic belief is largely about signaling rather than truth, why do you often cite academic consensus to support your own positions, without adding a disclaimer like "but of course academic belief is mostly about signaling"? (See here and here for a couple of recent examples.) Do you expect that most of your readers are already familiar enough with signaling theory and how it applies to academia, that they can take this into account and not over-react to the evidence of academic consensus that you provide, even without an explicit disclaimer?
Robin: I have no problem with expected values or utilities (at least in theory; in practice, there can be lots of difficulties in the implementation). But if you look at Caplan's post that you link to, there's nothing about E and nothing about U. It's all about P. Here's Caplan:
The P(minimum wage causes unemployment|Card/Krueger study's findings). Suppose P(CK findings|m.w. does cause unemployment)=.3, P(CK findings|m.w. does not cause unemployment)=.8, P(m.w. does cause unemployment)=.99, and P(m.w. does not cause unemployment)=.01. Then the conditional probability comes out to .3*.99/(.3*.99+.8*.01)=97.4%.
I just don't think this particular kind of reasoning makes sense or is a helpful way to do statistics. The world (at least, the social science world) just isn't discrete in this way.
Anyway, you're free to disagree with this--many statisticians do. I just wanted to isolate the way in which what Caplan (and many others) think of casually as rational Bayesiansm is not something that fits in well with my conception of Bayesian statistics.
Robin: the elimination of minimum wage would have social as well as directly economic consequences. These social consequences could give rise to different economic conditions.I might expect that E[U|MW] – E[U|noMW & Obama gives a speech to business owners] would be different from E[U|MW] – E[U|noMW & Obama gives a speech to the unemployed] or from E[U|MW] – E[U|noMW because they just stop enforcing it]
In that way, having a specific policy change in mind gives us more reliable probability estimates.
Also, I sincerely doubt that politicians' decisions regarding minimum wages actually emerge from economic arguments (as opposed to: individual interest, political interest or sponsors, desire to appear strong/clever, pressure from party leaders). Richard, if you have some evidence of this, I would be interested to see it.
Seems to me Caplan is talking about an expected value: E[U|MW] - E[U|noMW]. I can't see the point of a stat language that forbids discussing estimating such things.
I took a look at Caplan's entry linked to above, and I can clarify that, as a practicing Bayesian statistician, I don't think that statements of the form "Pr (hypothesis is true)" are very useful. (In Caplan's example, he discussed "Pr (minimum wage causes unemployment)." I think you're better off having a model of the effects of proposed policy changes (for example, an increase or a decrease in the minimum wage) and estimating the effects from there. I agree with Caplan that there can be a place for an economist's prior distribution to enter here, but it can also make sense to try to understand what can be learned from a particular experiment. In that sense, I think things have gone the way Caplan would like. Despite the cited minimum wage study, and despite the overwhelming popularity in polls for raising the minimum wage by a lot, Congress has been reluctant to do much in that direction. Perhaps some of this reluctance can be attributed to the persuasive arguments of economic theory.
Getting back to my earlier point, I think you should be very wary of going from the reasonable statement that people should reason logically to the highly flawed (in my opinion) statistical procedure of assigning probabilities to discrete hypotheses and trying to do Bayesian inference from there. If you want to do Bayesian inference, I think you're better off building models, with all the costs and beneifts that has.
And, no, in response to the other comment above, I am not trying to "posture" or "police." As a blogger myself, I very much appreciate when people share their thoughts and comment on my blog, and I'm pretty sure that Robin feels the same way. Like "relativity" or "evolution," he word "Bayesian" gets used in lot of ways and I think it can be helpful to understand its technical meanings.
Professor Gelman,Although I think you offer great insights, I think you're also being a bit parasitic in your posturing relative to Prof. Hanson.
Prof. Hanson seems to me to be in good faith reaching for ways to optimize our social epistemology, whereas it seems to me that you're sniping at the margins here, in a way "seems a distraction from the main point of this post"
It feels to me like your policing is in the direction of reducing cognitive ambition, that I intuit is to our detriment.
From the point of view of this alternative decision theory are the patterns Bryan describes, of having no opinion until something is proven or of moving estimates all the way to each new empirical estimate ignoring the previous estimates and other data, rational? If not, that seems a distraction from the main point of this post.
Robin, I thought that your signalling theory was designed to explain why rankings of prestigious institutions doesn't correlate well with what independent rankings we have of the ability of individuals in those institutions. Am I wrong?
Eliezer, I suspect Wei Dai meant "profit-seeking", not rent seeking. Generally speaking, rent-seeking behavior does not increase social welfare.
Wei, I disagree that academia primarily harnesses status-seeking to produce knowledge. I think rather that academia is more of a coordination among knowledge seekers to optimize their knowledge-seeking. In other words, I think the credentialing of third party non-academics, etc. is to fund the knowledge-seeking, with some drift towards status driven behavior, like graduation ceremony pomp and circumstance. Faust is the academic archetype, it seems to me, like Midas is for the capitalist and Narcissus for the artist. Academia isn't the efficient route to status. I think status-games represent a corruption of the academic, or a concession to the non-academic world.
"Rent-seeking" has a particular meaning in economics. Is there a special economic term for ordinary-desire-for-money or do we just say "greed"?
Much of this is simply due to entrenched models of teaching. So, pretty much all intro econometrics courses assume classical frequentism, and the standard BLUE theorems for such workhorse methods as OLS all assume a CF framework. It is only at a much more advanced and specialized level that one is likely to encounter Bayesian methods, particularly in time-series analysis. This is in contrast to medical research, where I understand that Bayesian methods have come close to beating out the CF ones.
Of course, many economists when confronted with the more general arguments will grant that Bayesianism has a lot of appeal in principle. But as they only known CF models, they kind of let it go, sort of a "we are all Bayesians now, but only in theory for most of us" view.
It does seem that academic beliefs tracks truth better than most expressed human opinions, even though they're both about signaling.
Markets (ordinary ones) harness rent-seeking to improve social welfare. Academia seems to harness status-seeking to improve knowledge, but perhaps in a way that we don't understand (certainly not as well as we understand markets). Prediction markets would harness rent-seeking to improve knowledge, but interest in such institutions seem too low to make them a viable alternative to academia.
Is anyone working on trying to better understand academia and perhaps improving it on the margins, while leaving its signaling function intact?
Most expressed human opinions are largely about signaling rather than truth. If we had a way to find what folks with expert level knowledge thought that focused mainly on a truth motivation, we'd love to defer to that. But absent prediction markets, that seems hard to find.
Robin, if you believe that academic belief is largely about signaling rather than truth, why do you often cite academic consensus to support your own positions, without adding a disclaimer like "but of course academic belief is mostly about signaling"? (See here and here for a couple of recent examples.) Do you expect that most of your readers are already familiar enough with signaling theory and how it applies to academia, that they can take this into account and not over-react to the evidence of academic consensus that you provide, even without an explicit disclaimer?
Robin: I have no problem with expected values or utilities (at least in theory; in practice, there can be lots of difficulties in the implementation). But if you look at Caplan's post that you link to, there's nothing about E and nothing about U. It's all about P. Here's Caplan:
The P(minimum wage causes unemployment|Card/Krueger study's findings). Suppose P(CK findings|m.w. does cause unemployment)=.3, P(CK findings|m.w. does not cause unemployment)=.8, P(m.w. does cause unemployment)=.99, and P(m.w. does not cause unemployment)=.01. Then the conditional probability comes out to .3*.99/(.3*.99+.8*.01)=97.4%.
I just don't think this particular kind of reasoning makes sense or is a helpful way to do statistics. The world (at least, the social science world) just isn't discrete in this way.
Anyway, you're free to disagree with this--many statisticians do. I just wanted to isolate the way in which what Caplan (and many others) think of casually as rational Bayesiansm is not something that fits in well with my conception of Bayesian statistics.
Robin: the elimination of minimum wage would have social as well as directly economic consequences. These social consequences could give rise to different economic conditions.I might expect that E[U|MW] – E[U|noMW & Obama gives a speech to business owners] would be different from E[U|MW] – E[U|noMW & Obama gives a speech to the unemployed] or from E[U|MW] – E[U|noMW because they just stop enforcing it]
In that way, having a specific policy change in mind gives us more reliable probability estimates.
Also, I sincerely doubt that politicians' decisions regarding minimum wages actually emerge from economic arguments (as opposed to: individual interest, political interest or sponsors, desire to appear strong/clever, pressure from party leaders). Richard, if you have some evidence of this, I would be interested to see it.
Seems to me Caplan is talking about an expected value: E[U|MW] - E[U|noMW]. I can't see the point of a stat language that forbids discussing estimating such things.
I took a look at Caplan's entry linked to above, and I can clarify that, as a practicing Bayesian statistician, I don't think that statements of the form "Pr (hypothesis is true)" are very useful. (In Caplan's example, he discussed "Pr (minimum wage causes unemployment)." I think you're better off having a model of the effects of proposed policy changes (for example, an increase or a decrease in the minimum wage) and estimating the effects from there. I agree with Caplan that there can be a place for an economist's prior distribution to enter here, but it can also make sense to try to understand what can be learned from a particular experiment. In that sense, I think things have gone the way Caplan would like. Despite the cited minimum wage study, and despite the overwhelming popularity in polls for raising the minimum wage by a lot, Congress has been reluctant to do much in that direction. Perhaps some of this reluctance can be attributed to the persuasive arguments of economic theory.
Getting back to my earlier point, I think you should be very wary of going from the reasonable statement that people should reason logically to the highly flawed (in my opinion) statistical procedure of assigning probabilities to discrete hypotheses and trying to do Bayesian inference from there. If you want to do Bayesian inference, I think you're better off building models, with all the costs and beneifts that has.
And, no, in response to the other comment above, I am not trying to "posture" or "police." As a blogger myself, I very much appreciate when people share their thoughts and comment on my blog, and I'm pretty sure that Robin feels the same way. Like "relativity" or "evolution," he word "Bayesian" gets used in lot of ways and I think it can be helpful to understand its technical meanings.
Ha!I attempt one on my blog. I think agent persistence optimization is the model of rational behavior of an agent.
Professor Gelman,Although I think you offer great insights, I think you're also being a bit parasitic in your posturing relative to Prof. Hanson.
Prof. Hanson seems to me to be in good faith reaching for ways to optimize our social epistemology, whereas it seems to me that you're sniping at the margins here, in a way "seems a distraction from the main point of this post"
It feels to me like your policing is in the direction of reducing cognitive ambition, that I intuit is to our detriment.
We have a saying in statistics: "The shitty is the enemy of the good."
From the point of view of this alternative decision theory are the patterns Bryan describes, of having no opinion until something is proven or of moving estimates all the way to each new empirical estimate ignoring the previous estimates and other data, rational? If not, that seems a distraction from the main point of this post.