Science ROI Hype
Years ago as a researcher at NASA Ames, I considered returning to grad school. Thinking about where I might study prediction markets as applied to academia, economics of science looked promising, especially as Paul David headed an econ of sci group nearby at Stanford. But reading the literature I got a bad feeling – authors seemed to be dishonestly trying to help research agencies justify funding. So I instead when to Caltech to do experimental econ, whose results I trusted more. My distrust is confirmed in a recent three page Nature article:
Spending on science is one of the best ways to generate jobs and economic growth, say research advocates. But … the evidence behind such claims is patchy.
The number one current rationale for extra research investment is that it will generate badly needed economic growth. … Heeding such arguments, governments in Germany, Sweden, Canada and Australia, as well as the United States, have increased research spending as part of stimulus packages … Beneath the rhetoric, however, there is considerable unease that the economic benefits of science spending are being oversold. … The problem, economists say, is that the numbers attached to widely quoted economic benefits of research have been extrapolated from a small number of studies, many of which were undertaken with the explicit aim of building support for research investment, rather than being objective assessments. … “Too much of what has been done, has been done as a process of advocacy.” …
In one of the bedrock papers in this field, Edwin Mansfield, the late University of Pennsylvania economist, estimated that academic research delivered an annual rate of return of 28% (E. Mansfield Research Policy 20, 1–12; 1991). The figure has been widely quoted ever since. But Mansfield reached this estimate by interviewing chief executives, asking them what proportion of their companies’ innovation was derived from university research and, in effect, demanding that they come up with a number. “He was asking an impossible question.” …
Measuring the ROI from research has proved tough, and has produced a wide range of values (see table). Some … [ask] what contribution did a dozen neuroscience grants received by the University of Cambridge in 1972 eventually make to drug development? Such efforts are complicated, however, by the difficulties of attributing credit for any given drug to the numerous research teams involved over time. … “It is fair to say that this is an analytical dead end.” …
This [PR] influence derives in part from the activities of US medical research lobbyists. An example is the 2000 report Exceptional Returns: The Economic Value of America’s Investment in Medical Research by … Mary Woodard Lasker Charitable Trust that advocated biomedical research spending. … The document estimated that the steep decline in cardiovascular deaths in the United States between 1970 and 1990 has an economic value of $1.5 trillion annually, and deduced that one-third of this — $500 billion a year — could be attributed to medical research that led to new procedures and drugs. … Robert Topel, … whose work was cited in the report, distances himself from some of its claims. “Probably only a little of the fall in the cardiovascular death rate has to do with surgery and beta-blockers,” he says. …
Research agencies have no interest in assessing the costs of research fairly, says Barry Bozeman, a science-policy specialist. … “Honest clients are in short supply” for research in this field, he says. “Most of them think they already have the answers, and want someone to find the numbers to prove them right.”