The bottom line question here, also alluded to by Eliezer’s recent postings, is whether we can do better than science as we know it? And the meta-question, how can society know in general if an institution like science, or for that matter the Dragon Institute, is the best way that things could be done?

Science is making genuine progress. I don’t think there is much question of this, although obviously it is more true in some fields than others. Would there be more progress if scientists spent more time listening to amateurs and responding to their ideas? Would there be more progress if scientific opinion were more open to new ideas and less wedded to the dominant paradigm? Or would this lead to an ineffectual expenditure of wasted effort chasing down numerous blind alleys?

IMO, the things that come out of formal academic and research institutions are often vastly over-valued. The “big ideas” which come from institutional think-tanks are often the “easy” part of whatever problem is trying to be solved, while the supporting technologies needed to make the idea viable are often vastly more complex. For example, prediction markets are very simple if you compare them to the tools needed to support them on the Internet (try counting the layers of abstraction between some XML market data and the prediction market itself, on a remote server). In my experience the real bottlenecks to progress are in the devil’s details, which aren’t obviously identifiable to laymen.

But then might the institutes play an important role? People listen to them. They are impressive. When public opinion itself is a barrier to progress (such as prediction markets under United States law), mightn’t the institutions have a valuable role to play?

Seems like the bottom line is, are Instituters successful in getting rid of dragons? Do they have a good track record? Were there more dragons in the past, and fewer today? If they are doing well, I (as an Oompa Loompa) would be inclined to forgive the various inefficiencies.

As far as biases, the main issue IMO is whether the larger society is correctly perceiving the contribution of do-something amateurs. If such amateurs are rare and it is hard to distinguish the ones who are likely to be successful, then again any bias against them is relatively unimportant. However if the Institute’s downplaying of the role of successful amateurs is significantly slowing down progress against dragons, then society’s bias is imposing real social costs. I suppose the hard part is knowing which is the case.

This is a parable about planet earth?

To give a concrete example, let’s say I claimed that we don’t currently understand how humans recognize letters and numbers, and my proof is how no one has written a program that beat OB’s captcha. Then, let’s say someone responded by pointing to the numerous journal articles in the cognitivie sciences attacking the problem of character and object recognition. In such a case, I would be correct: the articles do not substitute for the real-world failure of computers beat the captcha.

Robin_Hanson’s point, then, is that academia trivializes the real-world accomplishments, even as that is supposed to be their ultimate goal, and proof, wherever they happen, of the academics’ inadequacy.