How Exceptional Is Gelman?

In response to my saying:

Academia is primarily an institution for credentialling folks as intellectually impressive, so that others can affiliate with them.

Andrew Gelman penned “Another reason I’m glad I’m not an economist“:

That [Robin] would write such an extreme statement without even feeling the need to justify it (and, no, I don’t think it’s true, at least not in the “academia” that I know about) . . . that I see as a product of being in an economics department.

I responded:

I have posted many times here on [this]. … The standard idealistic [story] is that academics know useful and important things, things which students want to learn, media want to report, consulting clients want to apply, … These idealistic theories … have [these listed] detailed problems. … It seems far simpler to me to just postulate that people care primarily about affiliating with others who have been certified as prestigious.

Andrew answered:

College students prefer to be taught by profs who research, and hence ignore students more, yet students have little idea what their profs research. . . . There is relatively little relation between what profs teach, what profs research, and what students do after they graduate.

To which I reply: No way, dude! Our students … send me emails asking when I’m going to teach multilevel models and Bayesian statistics. … There is a strong connection between what I teach and what I research. And it’s my impression that they do use this stuff after they graduate. …

You might say: Fine, but Robin is talking about academia in general, not the Columbia statistics department in general. That I could buy … But . . . in his blog entry Robin appears to be skeptical of my claim that the customers who pay my salary “learn how to fit multilevel models.”

On why he is funded:

The state government of New York or the Heritage Foundation or whatever, … I assume they would like their conclusions to be research-based, to avoid negative unintended consequences and all the other things that we worry about when considering policies.

My primary focus is academia in general, and grad students are not “college students.”  I have consistently told both college and grad students that stat classes are among the most useful later in a non-academic career.   So I am happy to grant that Andrew may be an unusual exception.  Nevertheless, consider:

  1. Since I’ve granted that my story is contrary to what people usually say and assume, saying “I assume” on funding just isn’t much of a contrary argument.  “My impression” on students isn’t much better.
  2. Your funding patrons may like to see studies using your methods not because they predict better but because they and you are more prestigious.  Could you tell the difference?
  3. I suspect most of your students never much use the methods they learn from you later.  Some no doubt do use them.
  4. Employers may want to hire your students to use your methods not because those methods predict better because they are more prestigious, and people who can master them are just better overall.
  5. Last week I mentioned that fancy stat forecasts are consistently beat by simple moving averages; have you done field tests to see how well your students actually do using your methods, compared with simpler methods?
GD Star Rating
Tagged as: ,
Trackback URL:
  • Floccina

    6. There are much faster cheaper and easier ways to learn what you teach.

    Education is free to very cheap credentials are quote costly. In fact IMO some of the value of credentials come from them being costly (signaling).

    • I don’t know about campus security in the United States, but around here you can just sneak into any lecture in a university. For free.

  • josh

    Andrew is naive beyond all hope.

  • babar

    the institution has value as a center of communication between accredited experts.

  • Robin has clearly learned rule #1 of media notoriety, which is to make overly strong claims to heighten the sense of controversy. It is of course a far more reasonable point to say that academia, like almost all visible human activity, includes a component of signaling; it might even be reasonable to say that because of the difficulty in measuring intellectual production and transmission, signaling is more important (relative to the ‘idealistic’ purpose) in academia than in other fields (such as auto repair). But who would want to read such hemming and hawing and careful qualification–other than academics?

    Which is a large part of my point here. While I find this blog quite entertaining, and often insightful, Robin appears to be succumbing to exactly the celebrity temptation that academic cultures are designed to combat. Sure, academics signal. But they also care very, very much about maintaining quality standards so that people who are supposed to be experts don’t go around overclaiming and thereby miseducating their students, their colleagues, or key decision-makers in policy and business spheres. A young assistant professor at BYU has written on this topic quite a bit…check it out here , B

    As an aside, I don’t think Robin has his facts quite straight on the teaching-research relationship. In my field (business), the best researchers are also spectacularly popular teachers. More broadly, I understand that the correlation between teaching and research success is at worse 0, and quite possibly positive. See this article for a summary.

    I suspect the correlation is indeed positive in the top 20 business schools. Much of business research entails convincing a skeptical audience that you have something important to say, conveying the point, answering questions intelligently, and having them walk away feeling they have learned something. If you can’t teach, it is very difficult to get your research published, no matter how brilliant you are at the technical side.

  • Mike

    I suspect you two might be arguing about two different things.

    I presume Robin admits that *some* people go to college to learn, they go to the “best” schools to increase the chances to interact with, and benefit from, the most creative researchers, and they use the knowledge they gained. The fraction will be greater when you consider graduate programs. Setting aside subconscious motives or why we have the motives we are conscious of, *most* academics do what they do with an honest belief that it is important — either because it increases their understanding, or the collective understanding of humanity, or has practical value, or because of indirect values related to providing “education” (which goes far beyond the facts and theories learned in the classroom).

    I think Robin’s stance is these people play a small part in the system, which takes in millions of students every year, and which is financed to the tune of many many billions of dollars every year. So it’s interesting to ask, what are the motives of the majority — why do they do what they do?

  • Robin:

    First off, let me assure your readers that I am funded neither by the Heritage Foundation nor the state of New York. The quote that you pulled is not about “why I am funded” but rather a response to a remark you made in your earlier blog entry about what “foundations and governments want to promote the creation and spread of.”

    Second, Mike’s summary makes sense to me. What puzzles me is why you seem reluctant to accept that the students who take my class, who pay thousands of dollars a year to attend Columbia, who specifically want to find out when I am teaching multilevel modeling and Bayesian statistics, are doing so because they want to learn the material.

    Sure, maybe you’re right, maybe they don’t want to learn the stuff at all–but, as far as I’m concerned, the burden of proof is on you here. You yourself describe your ideas as “contrary to what people usually say and assume.” Maybe, just maybe, most of the people are right here.

    To put it another way, you’re speculating and I’m speculating. You have a theory involving evolution and signaling, I have a theory that people pay tuition for a good reason. When it comes to motivations of students, funders, etc., both of us are offering argument and anecdote. When it comes to your experiences with GMU students who’ve taken your classes, your anecdotes are more relevant; when it comes to Columbia students taking my classes, I think my anecdotes are more relevant.

    Finally, regarding the effectiveness of simple statistical models: that’s fine. If you take a look at my books, we have lots of simple statistical models. I spend a lot of time in class discussing these. The sorts of multilevel models that I fit are pretty simple too; they just vary by group, which makes them look complicated to people who aren’t used to them. So I’m not sure what you mean by “simpler methods.” Do you suggest that I only fit models where coefficients don’t vary by group? I do this all the time, moving to multilevel models when needed. Basically, I just about always try to do the simplest possible thing that will work. It just turns out that, in many cases, the simplest possible thing has problems. If you think that the article you linked to suggests that students shouldn’t be taking my class, I think you’re missing a lot.

    • Andrew, I’m not talking mainly about conscious motivations but about social functions; the pressures which shape institutions over the long run. I am trying to account well for all our data on academia as a whole, and not focused specifically on GMU or Columbia.

  • Pingback: Overcoming Bias : How Exceptional Is Gelman? « Enjoy Your Life()

  • Pingback: Andrew Gelman to Robin Hanson: You want *simple* forecasting methods? We do them, too. As a matter of fact, we do *all* kinds of forecasting methods at Columbia —both the complicated ones and the simpler ones. Come to my class, buddy. | Midas Oracle()

  • It seems like Dr. Gelman is refusing to engage the economic analysis. (For a reason that is somehow not easily communicated to Robin.) In terms of economics, I think the burden of proof is on Andrew. Consider that:

    1) There can be an order of magnitude difference between state school and private school tuition. Surely the quality of undergrad instruction does not vary by that much. It is possible that parents/students are paying up for incrementally better instruction, but paying for future relationships, earnings power & status seems more likely, to anyone I would think.

    2) Dr. Gelman’s book on multilevel models is available through Amazon for $31. I’m sure it is clear and well-written. It’s even possible that Andrew would answer questions from readers via email. So in what sense exactly are graduate students paying thousands of dollars a year to “learn the material”?

    Robin’s “simpler methods” question goes in the direction of a cheap shot though as complexity isn’t necessarily overfitting.

  • Pingback: RE: Andrew Gelman versus Robin Hanson | Midas Oracle .ORG()