Blind Winners

In the late 1980s, when I was a nobody working on my own on "hypertext publishing," our name then for what would become the web, I gave a talk on the subject to a computer research group at Xerox PARC.  I argued that academic progress was limited by our ability to find related research, and that the web might help.  The researchers responded that they had no problems whatsoever finding relevant research; they knew all the relevant people, who would personally tell them about any important work.  The chance that someone outside their social circle would write something worth reading was so small as to be negligible. 

This came to mind recently when Scott Aaronson commented:

At least in physics (the Nobel area I know best), I think that the Nobels already correlate pretty well with which discoveries future generations regard as important. (Partly just because the academy often waits for a future generation’s verdict!) If you wanted to find a domain where prediction markets would give a big marginal advantage over what people are doing currently, the science Nobels are not the first place I’d look.

In general I suspect that winners in any social arena underestimate problems with that arena’s evaluation processes.  After all, the people winners think most worthy of winning, they and their social allies, have in fact won.   And that arena’s usual procedures are validated by the fact that they, the winners, use them.

GD Star Rating
loading...
Tagged as: , ,
Trackback URL:
  • http://www.scottaaronson.com Scott Aaronson

    Robin, I make it a point to scan the quant-ph abstracts every weekday, specifically in case someone I’ve never heard of has posted a paper I want to read. The web and arXiv seem to me like the biggest advances in dissemination of research in more than century, and I wish everyone would hurry up already and switch entirely to web-based open-access journals.

    When it comes to the science Nobels, one assumes the committee is at least aware of any serious contender, but of course its evaluation function might still be systematically skewed. Do you have any specific examples (besides the ones I listed in my own post) of cases where you think the evaluation process failed?

  • http://amnap.blogspot.com/ Matthew C

    Robin, I make it a point to scan the quant-ph abstracts every weekday, specifically in case someone I’ve never heard of has posted a paper I want to read. The web and arXiv seem to me like the biggest advances in dissemination of research in more than century, and I wish everyone would hurry up already and switch entirely to web-based open-access journals.

    It seems like most of the new physics research I hear about is published on arXiv first. Also there are tons of commentaries on other papers on arXiv. Why aren’t other science fields doing this? Are physicists just smarter, or is it just a field with more tradition of openness than, say, economics or biology?

  • Eric Falkenstein

    The problem with the soft sciences is that success is generally measured in an ability to create a viable thread–real business cycle models, Heath-Jarrow-Morton interest rate models, Input-Output matrices–and these may grow over a generation or two before the original cohort dies out and people recognize they were all in a self-referential delusion. That is, Professor Jones develops the flux-capacitor, Smith generalizes it to continuous time, Johnson then adds for heteroskedasticity. They all congratulate themselves, but then someone like Chris Simms comes along and says, hey, this simple model (ARMA) is just as good, if not better, than your complicated model (large scale Macro Models). Oops. The thread dies as professors retire.

    Many ideas people were sure to have concrete and important applications have turned out to be dead ends (remember MV=PT? Turnpike theorems? Bellman’s equations? Continuous time modeling?).

    I think the key is that any really good idea in econ is rather short, easy to summarize. If you say, “read these books and you will see his genius”, or “his models created other models”, there is no point there, just good rhetorical skills. If you say, Lucas was good because he built these models were could estimate, I would say, great, and what models have we then created that allow us to better understand growth, fluctuations, or inflation, better than the simple idea that in the long run the quantity theory holds? In contrast, Spence’s model shows that asymmetric information creates a value to signaling because individuals will make choices reflecting their private information. It is a model, and a result, and is pretty terse.

  • http://profile.typekey.com/robinhanson/ Robin Hanson

    Matthew, a great question.

    Scott, I have suspicions, more in econ than in physics, but not any I want to post publicly at the moment.

  • http://profile.typekey.com/sprague/ sprague

    Same thing applies to celebrities, who often seem to know each other long before they are famous.