Overcoming Bias

Animal experimentation: morally acceptable, or just the way things always have been?

Following the announcement last week that Oxford University’s controversial Biomedical Sciences building is now complete and will be open for business in mid-2009, the ethical issues surrounding the use of animals for scientific experimentation have been revisited in the media—see, for example, here , here, and here.

The number of animals used per year in scientific experiments worldwide has been estimated at 200 million—well in excess of the population of Brazil and over three times that of the United Kingdom. If we take the importance of an ethical issue to depend in part on how many subjects it affects, then, the ethics of animal experimentation at the very least warrants consideration alongside some of the most important issues in this country today, and arguably exceeds them in importance. So, what is being done to address this issue?

In the media, much effort seems to be devoted to discrediting concerns about animal suffering and reassuring people that animals used in science are well cared for, and relatively little effort is spent engaging with the ethical issues. However, it seems likely that no amount of reassurance about primate play areas and germ-controlled environments in Oxford’s new research lab will allay existing concerns about the acceptability of, for example, inducing heart failure in mice or inducing Parkinson’s disease in monkeys—particularly since scientists are not currently required to report exactly how much suffering their experiments cause to animals. Given the suffering involved, are we really sure that experimenting on animals is ethically justifiable?

In attempting to answer this question, it is disturbing to note some inconsistencies in popular views of science. Consider, for example, that by far the most common argument in favour of animal experimentation is that it is an essential part of scientific progress. As Oxford's oft-quoted Professor Alastair Buchan reminds us, ‘You can’t make a head injury in a dish, you can’t create a stroke in a test tube, you can’t create a heart attack on a chip: it just doesn’t work’. Using animals, we are told, is essential if science is to progress. Since many people are apparently convinced by this argument, they must therefore believe that scientific progress is something worthwhile—that, at the very least, its value outweighs the suffering of experimental animals. And yet, at the same time, we are regularly confronted with the conflicting realisation that, far from viewing science as a highly valuable and worthwhile pursuit, the public is often disillusioned and exasperated with science. Recently, for example, people have expressed bafflement that scientists have spent time and money on seemingly trifling projects—such as working out the best way to swat a fly and discovering why knots form—and on telling us things that we already know: that getting rid of credit cards helps us spend less money, and that listening to very loud music can damage hearing. Why, when the public often seems to despair of science, do so many people appear to be convinced that scientific progress is so important that it justifies the suffering of millions of animals?

If You Snooze, You Lose

Jim Horne argues we get plenty of sleep:

The apparent desire for more shut-eye, together with oft-repeated assertions that our grandparents slept longer, all too easily leads to the conclusion that we in the west are chronically sleep-deprived. ... [Such] Claims ... are nothing new - in 1894, the British Medical Journal ran an editorial warning that the "hurry and excitement" of modern life was leading to an epidemic of insomnia.  ...

Over the past 40 years, there have been several large studies of how much sleep people actually get, and the findings have consistently shown that healthy adults sleep 7 to 7.5 hours a night. The well-known "fact" that people used to sleep around 9 hours a night is a myth. ...

Toilets Aren't About Not Dying of Disease

I'm not nearly as far gone as Robin on the idea that social status is the predominant human motivation, but here is a pretty powerful example from an interview with British journalist Rose George about her new book, "The Big Necessity: The Unmentionable World of Human Waste and Why It Matters" (I haven't read it, but I've read a bit about it, and Tyler Cowen recommends it).  Excerpt below the jump.

More Deafening Silence

From a review of medical licensing:

By almost all accounts, the quality of services consumers get from non-physician clinicians is at least on par with what they would get from a physician performing the same services. Dozens of peer-reviewed studies compare outcomes in situations where patients are treated by a physician, a physician assistant, or an advanced practice nurse.  Outcomes appear similar [15] - an important factor, considering that non-physician clinicians can provide many services at a much lower cost. ...

A review of more than 50 studies by the American Medical Association's Council on Medical Education found that the peer reviewed studies "almost uniformly conclude that . . . a non-physician clinician. . . can provide an acceptable level of care."  The Council did note that some observers find serious flaws in the literature, including small samples, lack of control subjects, and failure to control for differences in the severity of illness treated by physicians and non-physician clinicians. Nevertheless, physician groups are unable to point to studies showing worse health outcomes with mid-level clinicians.

Again a deafening silence.  Dozens of studies over decades consistently find no patient health differences between expensive docs and cheaper clinicians, but there is zero momentum to let clinicians replace docs - policy trends go the other way.  The "establishment" complains about flaws in existing studies, but feels no need to do better studies.  What is the point of social science if even relatively clear important results are ignored? 

Don't tell me this is just because the public hasn't heard about these studies - the media  covers lots of med news they think the public wants to hear.  My local paper has a whole weekly health section.  Obviously the media doesn't think the public wants to hear how they can drastically reduce med costs without sacrificing health.  Medicine is not about health. School is not about Learning.  FYI, here are some of those many studies: 

Noble Lies?

A New Scientist book review:

In the face of life's inconvenient facts - alcoholism, drug addiction, depression and craziness, to name a few - pseudoscientific medical concepts allow us to cast difficult moral problems as simple factual questions, readily soluble in the lab and in the hospital. Gary Greenberg's The Noble Lie is an impressive and fascinating round-up of such pseudoscientific notions and the ways in which they have come to count as genuine illnesses.

For instance, Greenberg explains how alcoholism's transition from vice to disease was a welcome one, especially following Prohibition. It was long viewed as an allergy, though the specific allergen persistently failed to appear. Even today, neither its disease-nature nor any possible cures have manifested themselves. Regardless, people are happy to accept the idea that addiction is a medical illness, perhaps, Greenberg suggests, because of our ambivalence towards the role of pleasure and our uncertainties about free will and self-determination. "With the disease model we have an answer," he writes, "one that has the imprimatur of science; addiction isn't wrong, it's sick."

Deafening Silence

Dan Klein harps on a too-rarely-asked question:

Counterparts to the FDA function in other countries. The FDA could adopt a standing policy that drugs permitted in Europe, Canada, Japan, Australia, etc. automatically become permitted in the US. Why doesn't the FDA adopt such a policy? Is it because a drug that is safe and effective in Australia, Canada, or France may not be safe and effective in the United States?

Alex Tabarrok, in the office next door, says he often harps on this and has never heard a reasonable counter-argument.  I'm especially fascinated by situations like this, where a devastating argument against a status quo policy is met by complete silence and disinterest.  Why don't intellectuals feel any need to respond to such arguments?  How can we make such silences more audible to more people?

Kids often analogously complain  "But Billy's parents let him do X."  And they should add, "What makes you think you know better than Billy's parents?"  Are we similarly just overconfident in the judgment of "our" regulators? 

Alex also harps on another tough question:

What can we learn about the need for efficacy requirements from the pervasive experience of off-label medicine, which has no FDA efficacy certification? The thrust of Tabarrok's argument is that off-label seems to work quite well and so why not drop efficacy requirements entirely?

Alex tells me the two best counter arguments are that off-label uses are often similar to on-label uses, and that drug firms have low incentives to run tests near the end of a drug's patent lifetime.  But then why not approve only similar on-label uses and approve all uses of unpatentable drugs (that have passed safety tests)? 

Beware High Standards

300,000 useless knee surgeries annually, costing $1.5B, continue in the US because a '02 study finding them useless "was criticized as flawed."  Now a new study finds the same results but "also has weaknesses."  We sure wouldn't want anyone to change beliefs or behavior because of a criticizable study - not when there are all those impossible-to-criticize studies out there. The news:

One of the most common surgical procedures performed in the United States -- arthroscopy to treat arthritis in the knee -- is useless. ... [The] findings are being published in today's issue of the New England Journal of Medicine. ... The study marks the second time a major study has questioned the operations, which can cost about $5,000 and are done on hundreds of thousands of Americans each year. In 2002, a study found the operations were no better than a sham procedure for arthritis. A year later, based on that and other studies, Medicare stopped paying for the operations for severe arthritis of the knee, and the number of claims for the procedure plunged.

But the 2002 study was criticized as flawed: All the operations were done by one surgeon, and the subjects, who were all men, tended to be older than the typical patient. As a result, many surgeons continued to perform the procedures. Feagan says that as many as a third of the estimated 950,000 arthroscopic knee surgeries performed each year in the United States are probably still being done for arthritis, which affects about 27 million Americans.  ...

Top Docs No Healthier

My two years a RWJF Health Policy Scholar exposed me to enough data to make me a skeptic on the marginal aggregate health value of medicine.  But where data is silent I try to give medicine the benefit of the doubt, such as in assuming average values are higher than marginal values, and that top med school docs give more value than others.   So I am shocked to report that in a randomized trial of 72,000 hospital stays by 30,000 patients, patients of top med school docs were no healthier:

The school affiliated with Program A is the top school in the nation when ranked by the incoming students' MCAT scores, and it is always near the top. In comparison, the lower-ranked program [B] that serves this VA hospital is near the median of medical schools. ... [Added: other ways A beats B here.] Patients treated by the two teams have identical observable characteristics and have access to a single set of facilities and ancillary staff. ...

Health outcomes are not related to the physician team assignment. ... Program B is associated with ... a 0.3 percentage-point reduction in 5-year mortality (or 0.6% of the mean).  ... The confidence interval is [-0.0162, 0.0106]. ...

Randomised Controlled Trials of Parachutes

It is tempting to react to unscientific methods of medical practice by rejecting any treatment that isn't supported by rigorous scientific evidence.  Here's a parody of naive implementations of evidence-based medicine that demonstrates the pitfalls of doing so:
Smith GCS, Pell JP. (2003). Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials. BMJ, 327(7429), 1459-1461.

From the paper:

Results We were unable to identify any randomised controlled trials of parachute intervention.

Conclusions As with many interventions intended to prevent ill health, the effectiveness of parachutes has not been subjected to rigorous evaluation by using randomised controlled trials. Advocates of evidence based medicine have criticised the adoption of interventions evaluated by using only observational data. We think that everyone might benefit if the most radical protagonists of evidence based medicine organised and participated in a double blind, randomised, placebo controlled, crossover trial of the parachute.

There are some interesting comments on the paper here and here.

Good Medicine in Merry Old England

Here's the abstract of an article by Martin, Rice, & Smith in the current issue of the Journal of Health Economics (generally regarded as the top journal in the field):

Empirical evidence has hitherto been inconclusive about the strength of the link between health care spending and health outcomes. This paper uses programme budgeting data prepared by 295 English Primary Care Trusts to model the link for two specific programmes of care: cancer and circulatory diseases. A theoretical model is developed in which decision-makers must allocate a fixed budget across programmes of care so as to maximize social welfare, in the light of a health production function for each programme. This yields an expenditure equation and a health outcomes equation for each programme. These are estimated for the two programmes of care using instrumental variables methods. All the equations prove to be well specified. They suggest that the cost of a life year saved in cancer is about £13,100, and in circulation about £8000. These results challenge the widely held view that health care has little marginal impact on health. From a policy perspective, they can help set priorities by informing resource allocation across programmes of care. They can also help health technology agencies decide whether their cost-effectiveness thresholds for accepting new technologies are set at the right level.

One shouldn't overstate the importance of this; it's only one study and it only deals with two medical conditions.  And of course the study was done on English data, not U.S. data.  We all know that there is evidence that the marginal unit of U.S. medicine has little or no health benefit, so this would be a noteworthy result if the study were done on U.S. data.  I don't know how noteworthy it is for English data.  Does anybody know if there is any RAND study type evidence about the effectiveness of the marginal unit of medicine in England or in other European countries?

When I was a kid, a cousin who lived in England came to visit us and showed me how to crack open those little plastic cubes containing the four one-use camera flashbulbs we had back then and set them off with a battery.  That totally rocked my world.  So as far as I'm concerned those guys are all geniuses.