Total vs. marginal effects, or, Are the overall benefits of health care “probably minor”?

I was having an interesting discussion with Seth Roberts about his claim that "the overall benefits of health care are probably minor."

The basis of his claim is evidence cited by Aaron Swartz:

In the 1970s, the RAND Corporation picked out 7700 people in six cities and gave half of them free health care. Those lucky ones took advantage of it (spending 30-40% more on average) and they spent it on reasonable things (as judged by medical observers), but they didn’t seem to get any healthier. . . . The RAND study was by far the biggest study of this kind, but other studies find similar results. One analysis found that regions whose Medicare programs give out more money (when the underlying healthiness of the residents is held constant) see no increase in survival rates. A replication found the same results in VA hospitals. Cross-national comparisons find “the impact of public spending on health is … both numerically small and statistically insignificant”. Correlational studies find “Environmental variables are far more important than medical care.” And there are more where that came from.

Several discussants (including myself) at Seth’s blog were skeptical about his skeptism, citing various successful medical treatments (in my case, fixing a compound fracture of the wrist; others mentioned cancer treatment, etc.).  Seth responded:

The RAND study, of course, is limited — but is there a better attempt to figure out the overall value of medicine? I don’t know of one. if you can point me to a study that shows the more-than-minor value of modern medicine I’d love to look at it. . . . when the overall effectiveness of medicine has come under scrutiny, it has not fared well — and the RAND study is a good example.

 

Total vs. marginal effects

I have not looked at the Rand study so can’t comment on the details, but my first thought is that the marginal benefits from additional health care will be less than the benefits from good existing care. So, even if more is not much better, that doesn’t mean that the overall benefits of existing care are “minor.”

From a policy standpoint, it is the marginal effects that are the most interesting, since nobody (even Seth?) is proposing to zero out medical treatment.  Presumably there are diminishing returns, and the benefit/cost ratio for additional treatment is less than that for existing treatment.  (And, indeed, some medical care can make things worse, even in expected value; for example, you can get catch the flu in the doctor’s waiting room.)  But, unless I’m missing something, Seth and Aaron are confusing marginal with total effects.

P.S.  Also see Robin Hanson’s discussion (with lots of links), which explicitly distinguishes between marginal and total effects.  Here I’m not expressing any position on the marginal effects of health care (given my ignorance on the topic), just pointing out that Robin’s position seems to have become overstated by others.

P.P.S.  Further discussion appears in the comments here.  In particular, take a look at Jake Bowers’s discussion of how to interpret the Rand experiment.

GD Star Rating
loading...
Tagged as:
Trackback URL:
  • http://yorkshire-ranter.blogspot.com/ Alex

    Vital unspoken assumptions – the alternative is “good existing care” (does this hold for most situations where this study is called in evidence? I doubt it), the only benefits from free health care are medical (sickness is the leading cause of personal bankruptcies in the US), health care has benefits pre-sickness (that two representative samples of the same population got sick at the same rate is obvious and trivial – what matters is how well they recovered. whether you like it or not, medicine is biased to cure rather than prevention).

    Further, applying some principles of Overcoming Bias here, does Robin, who loves to cite this study, believe that health care has no benefit to him? Would he be willing to go without? If not, where comes the arrogance to suggest that everyone else should have none, because they’ll only waste it?

  • http://byrneseyeview.com Byrne

    …the only benefits from free health care are medical (sickness is the leading cause of personal bankruptcies in the US)

    Well, paying for sickness may be the leading cause (but I think that’s just because it’s more volatile; if you have one expense that’s a steady $30,000/year, and one that’s $1,000 a year most years and $20,000 once a decade, the second will be correlated with bankruptcy even though it’s a smaller fraction of your lifetime budget). But the RAND study demonstrates pretty clearly that, as of 1970, the marginal benefit of medicine was 0. So, assuming medical spending has increased since then, this seems to imply that we have 30 years of increased spending whose benefits are a lot lower than we might assume.

    Perhaps medical care and medical research has completely reformed itself since then, and the drastic increases in medical spending which happen not to correlate with increased health are just a coincidence that in no way confirms the study’s conclusion that increases in medical spending don’t correlate with increased health. We could test this (again!) by taking two similarly large groups, giving one group discounted medical care paid for by having the other group pay a premium, but I’d rather keep it in mind when considering new medication or medical procedures.

  • http://profile.typekey.com/robinhanson/ Robin Hanson

    Andrew and Alex, I’m not proposing cutting medicine to zero, but then I also can’t say we have much evidence that we couldn’t cut very deep with little or no loss. I posted on what the RAND experiment says about the average value of medicine, in addition to what it says about the marginal value. Personally I try to cut back by 50-80% relative to what I would have done before knowing all this.

  • http://yorkshire-ranter.blogspot.com/ Alex

    Are you sure that “health” and “life expectancy” haven’t decorrelated? It seems quite plausible that much medical spending’s chief effect is to keep codgers going longer in the face of more and more complex ailments, which would neatly resolve the problem.

    Robin, in what way have you “cut back”? If you are British, that suggests you’ve decided to evade 50-80% more of your income tax.. (and if you are and have cut back on private insurance, well, the marginal benefit of private health insurance in a country with universal provision probably is very close to zero)

    More broadly, I reckon this is an artefact of the definition of “health”. Longer life expectancy with no better health, which is what I suspect is happening, may not be ideal and indeed I feel quite cynical about it. But then, I have sufficient self-awareness to realise that this is an easy thing to think when you’re 26.

  • http://profile.typekey.com/sentience/ Eliezer Yudkowsky

    Everyone please remember that a small NET statistical effect does not necessarily indicate small GROSS causal effects. A large benefit of medicine may be counterbalanced by medical errors, deaths under general anesthesia or from unnecessary surgery, diseases caught in the hospital, unknown side effects of drug combinations… many are talking as if the bold hypothesis under consideration is that medicine is a null-op. This is not the bold hypothesis. The bold hypothesis is actually a spectrum of hypotheses under which the marginal benefits of medicine and its marginal harms range in unison from low to high – any point in this spectrum accounts for the RAND data. For every bright anecdote that pops up in your mind about how modern medicine cured your uncle’s skin cancer (which I’m quite willing to accept it did) consider the anecdote someone else posted to this thread about an uncle who went under general anesthesia for surgery to his knee and never woke up.

  • http://profile.typekey.com/robinhanson/ Robin Hanson

    Alex, most of the correlations studies that find no effect study mortality. I live in the US, use Kaiser, and limit my visits.

  • Doug S.

    The RAND study doesn’t seem like a study on the overall benefits of “medicine” – what it compared was not “medicine” vs “nothing” but “free medicine” vs “expensive medicine”. In other words, making people pay for their own health care didn’t seem to make them sicker. If someone gets appendicitis, they go to the emergency room and get treatment regardless of whether they’re insured or not. If something is well-known to be effective for an obvious problem, people are still going to get treatment even if they have to pay for it – or they may end up as someone who gets treated but defaults on the hospital bill. I can’t imagine that people in the RAND study were left to die of appendicitis if they couldn’t pay out-of-pocket for surgery, or that they didn’t get the standard vaccinations. The marginal benefit of giving people health care beyond what they would pay for themselves may be very small, but if they really need it, people often find a way to get it anyway.

  • Hopefully Anonymous

    Doug, good point about emergency room health care. Since there seems to me to be an implied criticism of universal health care programs in this discussion, I think it’s worthwile to point out that the people in the RAND study who were given comprehensive health coverage may have overall spent less social resources than the ones who didn’t and may have relied on emergency room care instead, even if those relying on emergency room care may not have suffered a loss of lifespan as a result. And a more efficient health care system could in theory free up more resources to, for example, invest in further public health innovation (or innovation in any other area such as SENS) that could work to increase our healthy lifespans.

  • http://profile.typekey.com/bayesian/ Peter McCluskey

    Alex, Robert Fogel’s book The Escape from Hunger and Premature Death, 1700-2100 provides evidence that the fraction of time that people suffer from illness has decreased significantly over the past century. Fogel presents a nutrition-related hypothesis to suggest why this trend will continue for a while longer.
    It may be that much medical spending is devoted to extending life but not health, but I doubt you can find much effect of that on overall time spent in poor health.

  • joe

    “but then I also can’t say we have much evidence that we couldn’t cut very deep with little or no loss.”

    Robin,
    You also don’t have any evidence from the Rand study which suggests that you could “cut very deep with little or no loss.” As you have said before, the Rand study is about the marginal benefits of more medicine on top of what you would have received if you had to pay for it yourself.

    Increased healthcare usage with no signficant positive benefit is not equivalent to suggesting that we could significantly decrease medicine usage without lowering health.

  • http://yorkshire-ranter.blogspot.com/ Alex

    This is interesting:

    According to a recent RAND study, the uninsured only get about 53.7 percent of the care that experts say they should get. But what’s stunning is that patients with private, fee-for-service insurance are “even less likely to receive the proper care.” Likewise, it’s true that lack of health insurance causes about 18,000 premature deaths annually. But about 98,000 people are killed each year by medical errors, and 126,000 die “from their doctor’s failure to observe evidence-based protocols for just four common conditions: hypertension, heart attacks, pneumonia, and colorectal cancer.”


    The root problem here, Longman argues, is that doctors tend to perform lots and lots of unnecessary treatments–partly because they get paid per service, and have no real economic interest in their patients’ long-term well-being. Dr. Elliot Fisher, a Dartmouth researcher, estimates that “30 percent of all Medicare spending goes for unnecessary operations and procedures.” (Notably, an oversupply of specialists will often create its own demand: Per capita Medicare costs for terminally ill patients in Miami are $50,000 more per year than in Minneapolis, but patients in Miami don’t live any longer–the city’s high concentration of specialists are just providing more treatments that simply aren’t needed.)..

  • http://profile.typekey.com/robinhanson/ Robin Hanson

    Joe, the RAND study looked at a 30-40% “margin”, and we have lots of other correlation studies, where spending often varies by a factor of two or more.

  • joe

    The point is that a net-zero benefit of a 30-40% marginal increase does not mean that you can reduce your current levels of meds without losing benefits. No gain from increasing does not equal the ability to decrease much from current spending without losing much in health benefits.

  • http://profile.typekey.com/sentience/ Eliezer Yudkowsky

    New York magazine anonymously interviews five doctors:

    Dr. Lung: Research is a real problem. Doctors just make up the data. They don’t report negative side effects, no question about it. I used to write the results on my reports that were negative and nobody printed them. Only if it’s positive does it get published in a journal. A doctor I know used to publish papers like nobody’s business, and all the doctors who came and left told me he made up data to satisfy NIH grants and pharmaceutical grants. He was and still is very popular.

    And we’ve already heard that two thirds of the “statistically significant” reports don’t replicate, negative papers don’t get published, multiple studies are done, and nobody bothers with replications anyway. And the ability to get bogus results by fancy statistics is so severe that the control variable weightings of papers that were supposed to be looking at something else are consistent among themselves, but completely out of whack with the papers that are supposed to be investigating the main result.

    Perhaps we should not be so puzzled that we have all these wonderful papers showing individual medical procedures as effective, and yet in the aggregate, marginal extra spending on them has a (net!) result of nothing.

  • http://profile.typekey.com/sentience/ Eliezer Yudkowsky

    The quoted paragraph was:

    “Dr. Lung: Research is a real problem. Doctors just make up the data. They don’t report negative side effects, no question about it. I used to write the results on my reports that were negative and nobody printed them. Only if it’s positive does it get published in a journal. A doctor I know used to publish papers like nobody’s business, and all the doctors who came and left told me he made up data to satisfy NIH grants and pharmaceutical grants. He was and still is very popular.”

    Everything else is me.

  • albatross

    Eliezer:

    Is there evidence that increasing medical spending actually makes people worse off, or just that it stops having a benefit?

    If there’s a point where spending more actually makes you die earlier, then we’re presumably in a world where there’s a rate of medical error that kills you, and a diminishing rate of improved survival from medical care. When the two curves cross is the optimal benefit from medicine; after that, you’re getting marginal care and dying of it. (Think of purely cosmetic procedures here.)

    If the response goes flat at some point (more spending has no effect on life expectancy, what I think the RAND study demonstrates within a certain range), then the model is a bit harder to understand. We know there’s a rate of medical error. Maybe professional judgement and regulation and fear of lawsuit chop out most of the procedures/treatments that have a higher rate of killing you than saving you?

    Anyway, which model is a better description of reality is very important, because it tells us where to focus energy. In the first model, regulators/payors/etc ought to be excluding the worst treatments. In some sense, that’s easy to do, because it’s just applying existing statistics and some level of judgement–you don’t need to do anything new.

    In the second model, that’s already been done, and all that’s left is to make the treatments more effective or safer. That’s worth doing, but is hard, in the sense that you actually have to do something clever and new; it’s a research and teaching and best-practices kind of problem, not a straightforward regulatory problem.

  • Hopefully Anonymous

    Eliezer, great June 15, 2007 at 02:11 AM post. Perhaps there should be Replication prizes, for people who do the best replication studies each year (that succeed or fail in replicating a “statistically significant” study), and perhaps there should an investigative prize for people who expose falsified data by a scientist. I can think of few areas where we have a greater stake in overcoming bias than in cutting edge biomedical research of the type likely to extend healthy lifespan.

  • current grad student

    Eliezer,

    Health insurance level was randomized, actual medical treatment was not. Thus one should be careful when making causal inferences regarding increased medical usage in this study.

    I admit to the existence of the biases you bring up in regards to the medical literature, but you can’t be so quick to dismiss other explanations when people were in control of making decisions regarding the actual amount of medical treatment they received.

    In addition, an increase in doctor and ER visits is not equivalent to an increase in medical treatment. Not every time you go to the doctor do you receive medicine. Also, people with less of a cost to seeing a doctor are likely to go to the doctor for less severe conditions than those who must weigh the financial cost against their perceived need. We can go on and on thinking of other explanations in this study… such is often the heavy burden in an observational study… confident conclusions cannot always be made when other logical conclusions are plausible.

    Aside, with a mean age of 31, how much of a benefit measured in terms of overall health should one have expected to see over a 3-5 year period?