Academic Tool Overconfidence?

In the November Scientific American, Stuart Kauffman offered a bias theory to explain why economists show little interest in his "complexity" economics: 

Unexpected change bedevils the business community endlessly … Economists have so far not been able to offer much help to firms trying to be more adaptive. Although economists have been slow to realize it, the problem is that their attempts to model economic systems focus on those in market equilibrium or moving toward it. … The path to maximum prosperity will depend on finding ways to build economic systems in which new niches will generate spontaneously and abundantly. Such an approach to economics is indeed radical. It is based on the emergent behavior of systems rather than on the reductive study of them. It defies conventional mathematical treatments because it is not prestatable and is nonalgorithmic. Not surprisingly, most economists have so far resisted these ideas. Yet there can be little doubt that learning to apply these lessons from biology to technology will usher in a remarkable era of innovation and growth.

The claim seems to be one of academic tool overconfidence: academic communities are biased toward the conceptual tools they use the most, and against approaches that rely on other tools.  Now there are large coordination payoffs from academics using similar tools, since this lets them compare and build on each others’ results.  And this must put pressure on any given researcher to use tools similar to those used by others in his field.  But it is far from obvious to me that this constitutes a bias. 

GD Star Rating
loading...
Tagged as: ,
Trackback URL:
  • http://homepage.mac.com/redbird/ Gordon Worley

    To me it feels like there’s bias, at least in mathematics. Since I study graph theory, my proofs have a combinatorial style. To other graph theorists, these proofs are great, but when I try to provide a valid combinatorial style proof to a question an analyst poses, it’s dismissed as “heuristic argument” because it doesn’t follow the proof methods they prefer. Again, both proofs are correct, but because the methods are not the social norm, they may be rejected.

  • luis_enrique

    Well I like the idea that economics might be constrained by its maths, and I like the idea that the behaviour of biological systems could be very informative about economics (Paul Ormerod is another proponent of this idea). But having read this article, I’m not clear what he’s actually proposing as an alternative to economics as is. He talks about the value of diversity but that just sounds like the division of labour/returns to scale argument. What exactly are the alternative mathematical / modelling mechanisms and techniques that biologists are using that economics could adopt?

  • http://profile.typekey.com/robinhanson/ Robin Hanson

    Gordon, is it a bias that people who speak French find it hard to understand someone speaking Chinese?

  • http://www.jamesdmiller.blogspot.com/ James D. Miller

    Let’s say that because of the coordination benefits Robin describes economists rationally rely on a set of mathematical tools. It is very possible that these mathematical tools give economists a bias outlook on the economy. But the bias cost might be justified because of the coordination benefits of using the tools.

    Robin is claiming that because of coordination benefits economists are not obviously bias towards using their set of mathematical tools. But another important issue is whether using these tools gives economists a bias view of the economy.

  • http://profile.typekey.com/robinhanson/ Robin Hanson

    James, what bias would using math produce on economists’ outlook?

  • http://www.jamesdmiller.blogspot.com/ James D. Miller

    Robin,

    Economists are bias towards studying issues that can be modeled. For example, most introductory economics textbook spend little time talking about innovation even though innovation is extremely important to the economy. This is because economists don’t have a nice mathematical model of innovation. As a result, students who complete intro micro often don’t fully understand the importance of innovation to the economy.

    Also, consider a PhD student who is going to develop one of two models for his dissertation. The first model will be mathematically unsophisticated, but will offer slightly more insight into the underlying economic phenomenon (even when you include the coordination benefits you describe). The second model will be extremely mathematically sophisticated and elegant but will offer slightly less insight. If the student wants an academic career, wouldn’t you suggest he go with the second model?

    PhD students are well advised to signal high mathematical ability even if doing so reduces the explanatory power of their papers. This may all be rational because understanding mathematics is so important to understanding the economy so an economic department might wisely decide not to hire someone who hasn’t proved his mathematical abilities. But still this biases many economists away from working on issues that don’t subject themselves to deep mathematical analysis.

  • Stuart Armstrong

    Economists may be biased against certain mathematical tools because they don’t provide the sorts of answers that they want – as in precise, prescriptive answers. Stock market analysis, in my experience, is carried out not by economists, but by physicists. They have the proper probabilistic tools to be able to make money off stocks and options while being (most of the time) completly unable to predict whether the stock or the market will rise or fall.

    But only few businesses are interested in this sorts of analysis, so economists continue with their incomplete models, which at least give a clear “yes/no” answer to any question.

  • http://profile.typekey.com/willmcb/ will mcbride

    This all goes back to Adam Smith’s depiction of academia as fundamentally uncompetitive. Dan Klein has decribed the problem as a PhD circle, where the producers (of research) are also the consumers. Just as in any uncompetitive industry, progress is sluggish, and the name of the game is erecting barriers to entry. In academic economics this means absurdly complicated math models which provide little or no insight, and which harken back to the dawn of statistical modeling some 100 years ago.

    Dan Klein’s paper is here:

    http://www.econjournalwatch.org/main/intermedia.php?filename=KleinInvestigatingApril2005.pdf

  • http://profile.typekey.com/robinhanson/ Robin Hanson

    James, I agree that academics focus more on being impressive than on adding to insight. But this is feasible and happens for *any* standard tool; the fact that math models are among economists’ standard tools does not obviously make this problem worse.

    Will, entry into academia is far easier than in many other industries one would consider quite competitive. Yes academia seems inefficient at producing intellectual progress, but that is arguably not the product academia is selling.

  • TGGP

    Lachmann and Shackle tried to emphasize disequilibrium in economics, but that led them nearly to rejecting economics as a discipline. Schumpeter tried to analyze innovation, but for him they occurred in clusters as a result of inflation by banks, which seems to bear little resemblance to the continual innovation invested in by venture capitalists we see around us. Kirzner tried to improve upon that, but his doubt-free entrepreneurs who “see only profit” and simply cannot imagine not being successful hardly seems realistic either.*

    The piece reminded me of what Krugman called “Santa Fe syndrome”, and then when I checked to the bottom: surprise, surprise, he’s from the Santa Fe Institute!

    *This paragraph may contain some gross errors of ignorance of the history of economics.

  • http://cob.jmu.edu/rosserjb Barkley Rosser

    TGGP,

    Krugman was only a visitor for a short time about a decade ago at SFI. He is now at Princeton.

    More generally, I think that Kauffman is largely right, although there are variations on what is going on here. So, to be very simple-minded about it, one could say that SFI-type complexity people are more likely to use simulation models, sometimes based on genetic algorithms or other sorts of things, whereas more conventional theoretical economists are more likely to use axiomatic formal proof methods. The latter tend to focus on equilibrium outcomes, showing when they occur and their implications. The former are not necessarily equilibrium outcomes or processes, but may involve ongoing evolutionary dynamics and emergence, as Kauffman says.

    This is not necessarily ideological or preconceived, but it may be. Thus, we have the famous first Welfare Theorem, which states that a competitive, complete, and full information general equilibrium is Pareto optimal, that is “efficient.” While Austrians and some other pro-market economists, like Kirzner, do not like such analysis, others, especially those at Chicago, like Lucas or formerly Milton Friedman, did accept such arguments, liked them, and argued that they showed the superiority of free markets and so forth. The axiomatic mathematical tools used thus might be preferred for an ideological outcome, even though Walras was sort of a socialist, and such theorems can be used to justify interventions into markets by governments on certain grounds a la Musgrave and the “market failure” approach. So, a bit of a complex matter.

    Aside to Gordon Worley: are the Gordon Worley from Madison, Wisconsin that I used to know?

  • http://threadsafe.blogspot.com tc

    Is it a bias for economists to actually start believing in their models? If economists believe that their tools produce more credible results than other approaches (and why use them otherwise?) then they will look down on other fields e.g. sociology. Bewley’s book “Why Wages Don’t Fall During a Recession” basically did sociology (talking to a bunch of people), and concluded that many fancy mathematical models had nothing to do with reality… but did the economists who came up with them believe that they were true?

  • http://profile.typekey.com/robinhanson/ Robin Hanson

    tc, every method will produce conclusions that academics might be tempted to believe. Academics might well be overconfident to believe the conclusions of their methods, but it is not obvious that math methods produce more such mistakes.

  • Curt Adams

    Of course idea leaders will tend to be biased towards claims they are able to support. In order for an idea leader to meaningfully support a claim he has to be able to argue for it in public forums. Even if an economist personally believed that a, say, competing sociological model was just as likely as an economic model he’d be better off pushing the econ model because he can. Since this bias towards ideas you understand would have been advantageous as long as humans have been arguing ideas, I suspect there will be unconcious biases built in to our brains which result in favoring models we understand better without us needing to make the concious and cynical decision for that bias.

    In this specific case, I think the economists are right. I tried to extend Kaufmann’s model for the # of different cell types being a spontaneous self-organization due to the number of regulatory genes. In his model every gene affects the exact same number of other genes; I modified it to a stochastic model more like real regulatory networks. Spontaneous order collapsed – it required the mathematical peculiarities of his model. Indeed, it turns out that differentiation occurs via a small number of genes in a cascade of decisions, not via a spontaneous organization at all. I can’t think of any other biological models which came out particularly useful either. I haven’t followed Kaufmann recently but I’d be very surprised if he had any successful predictions in economics (he would have mentioned them), and it’s just quackery to say “Yet there can be little doubt that learning to apply these lessons from biology to technology will usher in a remarkable era of innovation and growth.” when the lessons are weak in biology and nonexistent in economics.

  • http://cob.jmu.edu/rosserjb Barkley Rosser

    Curt Adams,
    The Kaufmann model is not the main tool or action here. I would suggest you get your hands on either Vol. I or Vol. II of the books with the titles _The Economy as an Evolving Complex System_, #II edited by W.B. Arthur, S.N. Durlauf, and D.A. Lane, and #III edited by L. Blume and Durlauf, just out last year. These volumes, all proceedings of SFI conferences, show what is really going on.
    As for the Bewley book, it’s finding of the ubiquity of downward nominal wage rigidity underlies the presidential address just delivered a month ago at the AEA meetings in Chicago by George Akerlof, which takes this seriously as a foundation for builing macro models. What comes out looks a lot like old-timey “Keynesian” models found in 40-year old textbooks.

  • Curt Adams

    Barkley: My comment *was* addressed to Kaufmann’s models, because that’s what he was talking about. If you are aware of some other complexity theories that have shown predictive or engineering uses in biology or economics, I’m all ears. However, I think you should present some of this useful research and not just claim it exists. I’m highly skeptical of ideas that only get aired in sympathetic conferences.

    I was very interested in the complexity theory paradigm when I first encountered it but I was very disappointed by the lack of followup with evidence-based science, as well as my own discovery that one particular idea was very wrong. I would take Kaufmann’s speculations on spin glasses and economic groups far more seriously if there were evidence that intermediate control levels produced faster innovation (plus some other feature suggestive of spin glass dynamics like backtracking frequency) in some developing industry like, say, Chinese garment manufacturers. But, that idea’s been on the table for, what, over ten years, and, AFAIK, there’s no there there.

  • http://thinkingonthemargin.blogspot.com/ Brian Hollar

    I don’t think this constitutes a bias so much as the effects of investments of human capital in specific tools (convential economic methodology). I think economics is like any other field and that new techniques and tools take time to get build, adapt, and get accepted. I do think there may be structual issues within academia that make it resistant to rapid change (not all aspects of this are a bad thing — imagine if every new, untested theory was instantly accepted by everyone). I think it will take time to reorient the profession towards more adaptive models, assuming this line of research bears fruit. (There is also probably a public choice story to be told here as well.)

    I think the social sciences (including economics) may already trending in the direction of adaptive models. I see these tools as a compliment rather than a substitute for current economic methods. They may allow for a much better testbed for conventional (and unconventional) economic thinking.

    Coincidentally, I am taking a course in Agent Based Modeling (ABM) this semester with Robert Axtell (who recently came to GMU). It looks to be an exciting class and I am impressed with the applications of ABM to many fields of study. You can read more about ABM here.

  • http://cob.jmu.edu/rosserjb Barkley Rosser

    Curt,
    Oh, so you don’t like stuff produced in “sympathetic conferences,” eh?

    Blume, Lawrence, 1995, “The statistical mechanics of best-strategy response revision,” Games and Economic Behavior, 11:111-145.

    Brock, William A., 1993, “Pathways to randomness in the economy: Emergent nonlinearity and chaos in economics and finance,” Estudios Economicos, 8:3-55.

    Brock, William A. and Cars Hommes, 1997, “Rational routes to randomness,” Econometrica, 65:1059-1095.

    Brock, William A. and Steven N. Durlauf, 2001, “Discrete choice with social interactions,” Review of Economic Studies, 68:235-260.

    Dawid, Herbert, 1996, Adaptive Learning by Genetic Algorithms: Analytical Results and Applications to Economic Models. Springer-Verlag.

    Hommes, Cars H., 2006, “heterogeneous agent models in economics and finance,” in K.L. Judd and L. Tesfatsion (eds.) Handbook of Computational Economics, vol. 2, Amsterdam: Elsevier, pp. 1109-1186

    LeBaron, Blake, 2006, “Agent-based computational finance,” same book as Hommes paper above, pp. 187-233.

    However, do not think any of these specifically model your Chinese factory example, but plenty to chew on in them. For further references, you can go rooting around some of the papers on my website at
    http://cob.jmu.edu/rosserjb
    .

  • Curt Adams

    Thanks, that’s the kind of thing I was looking for. Of course I don’t care about a *particular* specific example, I just want some hard data.

  • TGGP

    I should note that I was actually unaware that Krugman was ever at Santa Fe, only that Kauffman was. I guess Krugman didn’t enjoy his stay there if he left shortly and named a “syndrome” after it.

    With regards to Chicagoites vs Austrians, it had seemed to me that the latter (of which I thought Kirzner was one, rather than “other”) were the most axiom-focused, with the former advocating “positivism” in economics so that theories would be evaluated by the correspondence of their predictions with reality rather than believing in results because they follow from true premises.

    I have seen economists expressing a low opinion of sociologists in other places. I think it is a tribute to the quality of this site that the same thing has not been displayed here. This is not to say that sociology deserves more respect than it gets, just that this is a classy place that doesn’t really go into dirt-slinging.

  • http://cob.jmu.edu/rosserjb Barkley Rosser

    TGGP,

    Krugman wrote a book inspired by his brief stay at SFI, _The Self Organizing Economy_. I have little to say about that book, although it does make some not totally inaccurate points.

    Regarding Chicagoites versus Austrians on the question of axiomatization, I would see it as more of an issue that cuts across both schools rather than one that divides the two from each other, although in general the Chicagoans were/are more favorable to using mathematics in their economic theorizing than were/are the Austrians, who tend to have a downright aversion to same.

    So, among Chicagoans, older ones such as Milton Friedman (the “positivist”) tended to eschew axiomatization, perhaps whom TGGP is thinking of, calling himself a “Marshallian” rather than a “Walrasian,” although he was hardly averse to at least some math, and did some very mathematical work for its time when he was an econometrician in his professional youth. More recent Chicagoans, such as Robert Lucas, are much more axiomatic and explicitly “Walrasian” in orientation, although Thomas Sargent became much less so after a visit to SFI in the 90s, which led him to write his _Bounded Rationality in Macroeconomics_, which argues that people actually have adaptive expectations, although learning processes tend to move them towards rational expectations asymptotically.

    With regard to the Austrians, the debate over “a priorism” is a central issue in the split between the “Misesians” and the “Hayekians,” although as near as I can tell, Robin Hanson seems to view both schools as ‘popular “nonsense”‘. In any case, it can be argued that Hayek was a proto-complexity SFI type, having published a famous essay back in 1967 on complexity in economics that echoes many themes that later would appear at Santa Fe, and which were already prevalent at such outfits as Brussels (the Prigogine group) and Stuttgart (the Haken group), both of which he contacted and interacted with, eventually becoming a close personal friend of Hermann Haken of Stuttgart, the founder of the synergetics approach.

  • http://profile.typekey.com/robinhanson/ Robin Hanson

    Barkley, I did not intend to state any claim about or evaluation of Austrian Economics; I just listed it as an example of a “heterodox” view, not officially favored by academia.

  • http://rafefurst.wordpress.com/ Rafe Furst

    While not the main thrust of the book, Eric D. Beinhocker’s Origin of Wealth makes a pretty good case for some form of academia bias.