I was having an interesting discussion with Seth Roberts about his claim that "the overall benefits of health care are probably minor." The basis of his claim is evidence cited by Aaron Swartz: In the 1970s, the RAND Corporation picked out 7700 people in six cities and gave half of them free health care. Those lucky ones took advantage of it (spending 30-40% more on average) and they spent it on reasonable things (as judged by medical observers), but they didn’t seem to get any healthier. . . . The RAND study was by far the biggest study of this kind, but other studies find similar results. One analysis found that regions whose Medicare programs give out more money (when the underlying healthiness of the residents is held constant) see no increase in survival rates. A replication found the same results in VA hospitals. Cross-national comparisons find “the impact of public spending on health is … both numerically small and statistically insignificant”. Correlational studies find “Environmental variables are far more important than medical care.” And there are more where that came from.
Health insurance level was randomized, actual medical treatment was not. Thus one should be careful when making causal inferences regarding increased medical usage in this study.
I admit to the existence of the biases you bring up in regards to the medical literature, but you can't be so quick to dismiss other explanations when people were in control of making decisions regarding the actual amount of medical treatment they received.
In addition, an increase in doctor and ER visits is not equivalent to an increase in medical treatment. Not every time you go to the doctor do you receive medicine. Also, people with less of a cost to seeing a doctor are likely to go to the doctor for less severe conditions than those who must weigh the financial cost against their perceived need. We can go on and on thinking of other explanations in this study... such is often the heavy burden in an observational study... confident conclusions cannot always be made when other logical conclusions are plausible.
Aside, with a mean age of 31, how much of a benefit measured in terms of overall health should one have expected to see over a 3-5 year period?
Eliezer, great June 15, 2007 at 02:11 AM post. Perhaps there should be Replication prizes, for people who do the best replication studies each year (that succeed or fail in replicating a "statistically significant" study), and perhaps there should an investigative prize for people who expose falsified data by a scientist. I can think of few areas where we have a greater stake in overcoming bias than in cutting edge biomedical research of the type likely to extend healthy lifespan.
Is there evidence that increasing medical spending actually makes people worse off, or just that it stops having a benefit?
If there's a point where spending more actually makes you die earlier, then we're presumably in a world where there's a rate of medical error that kills you, and a diminishing rate of improved survival from medical care. When the two curves cross is the optimal benefit from medicine; after that, you're getting marginal care and dying of it. (Think of purely cosmetic procedures here.)
If the response goes flat at some point (more spending has no effect on life expectancy, what I think the RAND study demonstrates within a certain range), then the model is a bit harder to understand. We know there's a rate of medical error. Maybe professional judgement and regulation and fear of lawsuit chop out most of the procedures/treatments that have a higher rate of killing you than saving you?
Anyway, which model is a better description of reality is very important, because it tells us where to focus energy. In the first model, regulators/payors/etc ought to be excluding the worst treatments. In some sense, that's easy to do, because it's just applying existing statistics and some level of judgement--you don't need to do anything new.
In the second model, that's already been done, and all that's left is to make the treatments more effective or safer. That's worth doing, but is hard, in the sense that you actually have to do something clever and new; it's a research and teaching and best-practices kind of problem, not a straightforward regulatory problem.
"Dr. Lung: Research is a real problem. Doctors just make up the data. They don’t report negative side effects, no question about it. I used to write the results on my reports that were negative and nobody printed them. Only if it’s positive does it get published in a journal. A doctor I know used to publish papers like nobody’s business, and all the doctors who came and left told me he made up data to satisfy NIH grants and pharmaceutical grants. He was and still is very popular."
New York magazine anonymously interviews five doctors:
Dr. Lung: Research is a real problem. Doctors just make up the data. They don’t report negative side effects, no question about it. I used to write the results on my reports that were negative and nobody printed them. Only if it’s positive does it get published in a journal. A doctor I know used to publish papers like nobody’s business, and all the doctors who came and left told me he made up data to satisfy NIH grants and pharmaceutical grants. He was and still is very popular.And we've already heard that two thirds of the "statistically significant" reports don't replicate, negative papers don't get published, multiple studies are done, and nobody bothers with replications anyway. And the ability to get bogus results by fancy statistics is so severe that the control variable weightings of papers that were supposed to be looking at something else are consistent among themselves, but completely out of whack with the papers that are supposed to be investigating the main result.
Perhaps we should not be so puzzled that we have all these wonderful papers showing individual medical procedures as effective, and yet in the aggregate, marginal extra spending on them has a (net!) result of nothing.
The point is that a net-zero benefit of a 30-40% marginal increase does not mean that you can reduce your current levels of meds without losing benefits. No gain from increasing does not equal the ability to decrease much from current spending without losing much in health benefits.
Joe, the RAND study looked at a 30-40% "margin", and we have lots of other correlation studies, where spending often varies by a factor of two or more.
According to a recent RAND study, the uninsured only get about 53.7 percent of the care that experts say they should get. But what's stunning is that patients with private, fee-for-service insurance are "even less likely to receive the proper care." Likewise, it's true that lack of health insurance causes about 18,000 premature deaths annually. But about 98,000 people are killed each year by medical errors, and 126,000 die "from their doctor's failure to observe evidence-based protocols for just four common conditions: hypertension, heart attacks, pneumonia, and colorectal cancer."
The root problem here, Longman argues, is that doctors tend to perform lots and lots of unnecessary treatments--partly because they get paid per service, and have no real economic interest in their patients' long-term well-being. Dr. Elliot Fisher, a Dartmouth researcher, estimates that "30 percent of all Medicare spending goes for unnecessary operations and procedures." (Notably, an oversupply of specialists will often create its own demand: Per capita Medicare costs for terminally ill patients in Miami are $50,000 more per year than in Minneapolis, but patients in Miami don't live any longer--the city's high concentration of specialists are just providing more treatments that simply aren't needed.)..
"but then I also can't say we have much evidence that we couldn't cut very deep with little or no loss."
Robin,You also don't have any evidence from the Rand study which suggests that you could "cut very deep with little or no loss." As you have said before, the Rand study is about the marginal benefits of more medicine on top of what you would have received if you had to pay for it yourself.
Increased healthcare usage with no signficant positive benefit is not equivalent to suggesting that we could significantly decrease medicine usage without lowering health.
Alex, Robert Fogel's book The Escape from Hunger and Premature Death, 1700-2100 provides evidence that the fraction of time that people suffer from illness has decreased significantly over the past century. Fogel presents a nutrition-related hypothesis to suggest why this trend will continue for a while longer.It may be that much medical spending is devoted to extending life but not health, but I doubt you can find much effect of that on overall time spent in poor health.
Doug, good point about emergency room health care. Since there seems to me to be an implied criticism of universal health care programs in this discussion, I think it's worthwile to point out that the people in the RAND study who were given comprehensive health coverage may have overall spent less social resources than the ones who didn't and may have relied on emergency room care instead, even if those relying on emergency room care may not have suffered a loss of lifespan as a result. And a more efficient health care system could in theory free up more resources to, for example, invest in further public health innovation (or innovation in any other area such as SENS) that could work to increase our healthy lifespans.
The RAND study doesn't seem like a study on the overall benefits of "medicine" - what it compared was not "medicine" vs "nothing" but "free medicine" vs "expensive medicine". In other words, making people pay for their own health care didn't seem to make them sicker. If someone gets appendicitis, they go to the emergency room and get treatment regardless of whether they're insured or not. If something is well-known to be effective for an obvious problem, people are still going to get treatment even if they have to pay for it - or they may end up as someone who gets treated but defaults on the hospital bill. I can't imagine that people in the RAND study were left to die of appendicitis if they couldn't pay out-of-pocket for surgery, or that they didn't get the standard vaccinations. The marginal benefit of giving people health care beyond what they would pay for themselves may be very small, but if they really need it, people often find a way to get it anyway.
Everyone please remember that a small NET statistical effect does not necessarily indicate small GROSS causal effects. A large benefit of medicine may be counterbalanced by medical errors, deaths under general anesthesia or from unnecessary surgery, diseases caught in the hospital, unknown side effects of drug combinations... many are talking as if the bold hypothesis under consideration is that medicine is a null-op. This is not the bold hypothesis. The bold hypothesis is actually a spectrum of hypotheses under which the marginal benefits of medicine and its marginal harms range in unison from low to high - any point in this spectrum accounts for the RAND data. For every bright anecdote that pops up in your mind about how modern medicine cured your uncle's skin cancer (which I'm quite willing to accept it did) consider the anecdote someone else posted to this thread about an uncle who went under general anesthesia for surgery to his knee and never woke up.
Are you sure that "health" and "life expectancy" haven't decorrelated? It seems quite plausible that much medical spending's chief effect is to keep codgers going longer in the face of more and more complex ailments, which would neatly resolve the problem.
Robin, in what way have you "cut back"? If you are British, that suggests you've decided to evade 50-80% more of your income tax.. (and if you are and have cut back on private insurance, well, the marginal benefit of private health insurance in a country with universal provision probably is very close to zero)
More broadly, I reckon this is an artefact of the definition of "health". Longer life expectancy with no better health, which is what I suspect is happening, may not be ideal and indeed I feel quite cynical about it. But then, I have sufficient self-awareness to realise that this is an easy thing to think when you're 26.
Andrew and Alex, I'm not proposing cutting medicine to zero, but then I also can't say we have much evidence that we couldn't cut very deep with little or no loss. I posted on what the RAND experiment says about the average value of medicine, in addition to what it says about the marginal value. Personally I try to cut back by 50-80% relative to what I would have done before knowing all this.
Eliezer,
Health insurance level was randomized, actual medical treatment was not. Thus one should be careful when making causal inferences regarding increased medical usage in this study.
I admit to the existence of the biases you bring up in regards to the medical literature, but you can't be so quick to dismiss other explanations when people were in control of making decisions regarding the actual amount of medical treatment they received.
In addition, an increase in doctor and ER visits is not equivalent to an increase in medical treatment. Not every time you go to the doctor do you receive medicine. Also, people with less of a cost to seeing a doctor are likely to go to the doctor for less severe conditions than those who must weigh the financial cost against their perceived need. We can go on and on thinking of other explanations in this study... such is often the heavy burden in an observational study... confident conclusions cannot always be made when other logical conclusions are plausible.
Aside, with a mean age of 31, how much of a benefit measured in terms of overall health should one have expected to see over a 3-5 year period?
Eliezer, great June 15, 2007 at 02:11 AM post. Perhaps there should be Replication prizes, for people who do the best replication studies each year (that succeed or fail in replicating a "statistically significant" study), and perhaps there should an investigative prize for people who expose falsified data by a scientist. I can think of few areas where we have a greater stake in overcoming bias than in cutting edge biomedical research of the type likely to extend healthy lifespan.
Eliezer:
Is there evidence that increasing medical spending actually makes people worse off, or just that it stops having a benefit?
If there's a point where spending more actually makes you die earlier, then we're presumably in a world where there's a rate of medical error that kills you, and a diminishing rate of improved survival from medical care. When the two curves cross is the optimal benefit from medicine; after that, you're getting marginal care and dying of it. (Think of purely cosmetic procedures here.)
If the response goes flat at some point (more spending has no effect on life expectancy, what I think the RAND study demonstrates within a certain range), then the model is a bit harder to understand. We know there's a rate of medical error. Maybe professional judgement and regulation and fear of lawsuit chop out most of the procedures/treatments that have a higher rate of killing you than saving you?
Anyway, which model is a better description of reality is very important, because it tells us where to focus energy. In the first model, regulators/payors/etc ought to be excluding the worst treatments. In some sense, that's easy to do, because it's just applying existing statistics and some level of judgement--you don't need to do anything new.
In the second model, that's already been done, and all that's left is to make the treatments more effective or safer. That's worth doing, but is hard, in the sense that you actually have to do something clever and new; it's a research and teaching and best-practices kind of problem, not a straightforward regulatory problem.
The quoted paragraph was:
"Dr. Lung: Research is a real problem. Doctors just make up the data. They don’t report negative side effects, no question about it. I used to write the results on my reports that were negative and nobody printed them. Only if it’s positive does it get published in a journal. A doctor I know used to publish papers like nobody’s business, and all the doctors who came and left told me he made up data to satisfy NIH grants and pharmaceutical grants. He was and still is very popular."
Everything else is me.
New York magazine anonymously interviews five doctors:
Dr. Lung: Research is a real problem. Doctors just make up the data. They don’t report negative side effects, no question about it. I used to write the results on my reports that were negative and nobody printed them. Only if it’s positive does it get published in a journal. A doctor I know used to publish papers like nobody’s business, and all the doctors who came and left told me he made up data to satisfy NIH grants and pharmaceutical grants. He was and still is very popular.And we've already heard that two thirds of the "statistically significant" reports don't replicate, negative papers don't get published, multiple studies are done, and nobody bothers with replications anyway. And the ability to get bogus results by fancy statistics is so severe that the control variable weightings of papers that were supposed to be looking at something else are consistent among themselves, but completely out of whack with the papers that are supposed to be investigating the main result.
Perhaps we should not be so puzzled that we have all these wonderful papers showing individual medical procedures as effective, and yet in the aggregate, marginal extra spending on them has a (net!) result of nothing.
The point is that a net-zero benefit of a 30-40% marginal increase does not mean that you can reduce your current levels of meds without losing benefits. No gain from increasing does not equal the ability to decrease much from current spending without losing much in health benefits.
Joe, the RAND study looked at a 30-40% "margin", and we have lots of other correlation studies, where spending often varies by a factor of two or more.
This is interesting:
According to a recent RAND study, the uninsured only get about 53.7 percent of the care that experts say they should get. But what's stunning is that patients with private, fee-for-service insurance are "even less likely to receive the proper care." Likewise, it's true that lack of health insurance causes about 18,000 premature deaths annually. But about 98,000 people are killed each year by medical errors, and 126,000 die "from their doctor's failure to observe evidence-based protocols for just four common conditions: hypertension, heart attacks, pneumonia, and colorectal cancer."
The root problem here, Longman argues, is that doctors tend to perform lots and lots of unnecessary treatments--partly because they get paid per service, and have no real economic interest in their patients' long-term well-being. Dr. Elliot Fisher, a Dartmouth researcher, estimates that "30 percent of all Medicare spending goes for unnecessary operations and procedures." (Notably, an oversupply of specialists will often create its own demand: Per capita Medicare costs for terminally ill patients in Miami are $50,000 more per year than in Minneapolis, but patients in Miami don't live any longer--the city's high concentration of specialists are just providing more treatments that simply aren't needed.)..
"but then I also can't say we have much evidence that we couldn't cut very deep with little or no loss."
Robin,You also don't have any evidence from the Rand study which suggests that you could "cut very deep with little or no loss." As you have said before, the Rand study is about the marginal benefits of more medicine on top of what you would have received if you had to pay for it yourself.
Increased healthcare usage with no signficant positive benefit is not equivalent to suggesting that we could significantly decrease medicine usage without lowering health.
Alex, Robert Fogel's book The Escape from Hunger and Premature Death, 1700-2100 provides evidence that the fraction of time that people suffer from illness has decreased significantly over the past century. Fogel presents a nutrition-related hypothesis to suggest why this trend will continue for a while longer.It may be that much medical spending is devoted to extending life but not health, but I doubt you can find much effect of that on overall time spent in poor health.
Doug, good point about emergency room health care. Since there seems to me to be an implied criticism of universal health care programs in this discussion, I think it's worthwile to point out that the people in the RAND study who were given comprehensive health coverage may have overall spent less social resources than the ones who didn't and may have relied on emergency room care instead, even if those relying on emergency room care may not have suffered a loss of lifespan as a result. And a more efficient health care system could in theory free up more resources to, for example, invest in further public health innovation (or innovation in any other area such as SENS) that could work to increase our healthy lifespans.
The RAND study doesn't seem like a study on the overall benefits of "medicine" - what it compared was not "medicine" vs "nothing" but "free medicine" vs "expensive medicine". In other words, making people pay for their own health care didn't seem to make them sicker. If someone gets appendicitis, they go to the emergency room and get treatment regardless of whether they're insured or not. If something is well-known to be effective for an obvious problem, people are still going to get treatment even if they have to pay for it - or they may end up as someone who gets treated but defaults on the hospital bill. I can't imagine that people in the RAND study were left to die of appendicitis if they couldn't pay out-of-pocket for surgery, or that they didn't get the standard vaccinations. The marginal benefit of giving people health care beyond what they would pay for themselves may be very small, but if they really need it, people often find a way to get it anyway.
Alex, most of the correlations studies that find no effect study mortality. I live in the US, use Kaiser, and limit my visits.
Everyone please remember that a small NET statistical effect does not necessarily indicate small GROSS causal effects. A large benefit of medicine may be counterbalanced by medical errors, deaths under general anesthesia or from unnecessary surgery, diseases caught in the hospital, unknown side effects of drug combinations... many are talking as if the bold hypothesis under consideration is that medicine is a null-op. This is not the bold hypothesis. The bold hypothesis is actually a spectrum of hypotheses under which the marginal benefits of medicine and its marginal harms range in unison from low to high - any point in this spectrum accounts for the RAND data. For every bright anecdote that pops up in your mind about how modern medicine cured your uncle's skin cancer (which I'm quite willing to accept it did) consider the anecdote someone else posted to this thread about an uncle who went under general anesthesia for surgery to his knee and never woke up.
Are you sure that "health" and "life expectancy" haven't decorrelated? It seems quite plausible that much medical spending's chief effect is to keep codgers going longer in the face of more and more complex ailments, which would neatly resolve the problem.
Robin, in what way have you "cut back"? If you are British, that suggests you've decided to evade 50-80% more of your income tax.. (and if you are and have cut back on private insurance, well, the marginal benefit of private health insurance in a country with universal provision probably is very close to zero)
More broadly, I reckon this is an artefact of the definition of "health". Longer life expectancy with no better health, which is what I suspect is happening, may not be ideal and indeed I feel quite cynical about it. But then, I have sufficient self-awareness to realise that this is an easy thing to think when you're 26.
Andrew and Alex, I'm not proposing cutting medicine to zero, but then I also can't say we have much evidence that we couldn't cut very deep with little or no loss. I posted on what the RAND experiment says about the average value of medicine, in addition to what it says about the marginal value. Personally I try to cut back by 50-80% relative to what I would have done before knowing all this.