Alex: I cannot think of a simpler change that would improve health care to as great an extent as freeing the data. He was riffing off Newsweek: In trying to find the oncologist or cancer center with the best track record on, say, stage IV bladder cancer, even the savviest patient quickly hits a wall: with a few exceptions, cancer centers treat these “outcomes” data like state secrets. … For these cancers there are indeed significant outcome differences depending where you are treated. … The Cleveland Clinic is the only one that makes its detailed outcomes data available to the public. … Although the National Comprehensive Cancer Network … collects data on how well its members adhere to treatment guidelines, it will not release the information on specific centers.
I agree that access to this data would be a good thing. However, I think your argument dismisses the data misinterpretation problem too quickly.
Here in the UK the media is constantly distorting debates about healthcare and as a result we have had to set up an independent body (NICE) to make comparisons between drugs and hospitals.
Data together with rational debate can leed to good outcomes.
Data, together with an irrational media, that falls for or exploits several cognitive biases (and errors in reasoning) in pursuing a story... doesn't leed to such great outcomes.
Access to information might have a negative effect in the short term and access to this information requires a more developed deliberative process within society.
But the right solution in my view is to have access to this information and provide better education for the public, journalists and better ways of presenting complex information. Independent oversight bodies (private or public) are a good idea too so long as they use a well thought through methodology.
Most procedures and situations are very standardized and statistics could be easily compared and understood by the general public. For example; all hospital induced infections should be public information. The rate of nosocomial infections (hospital created infections) says a lot about the sanitary conditions of the hospital and the quality of it's staff and management.
Things like outcome data for 4X heart bypasses by hospital and surgical teams can also be easily understood. You don't get a bypass unless you need it and you are in real trouble. In fact, I can't think of an area or problem where outcome data would not have been useful.
Even when asking a Dr. about outcome data, I have gotten answers like "very low risk of death" or 85% 2 yr cure rate but no references to journals or government collected statistics to back up those claims.
In a recent case (cardiac ablation), I was forced by lack of solid data to choose a Dr. on the basis of recommendations by reputation among other Dr.'s and the quality and readability of his scientific publications (which has nothing to do with how good his "hands" are).
It's not just that these stats reflect differing patient mixes, but that publicizing them would *cause* differing patient mixes. Hospitals would have a huge incentive to cherrypick their patients and refuse care to the sickest who needed it most.
My limited understanding is that this is already a problem in some areas of medicine, such as surgery. Don't surgeons routinely refuse to "accept the surgical risk" of operating on a patient who has severe health problems unrelated to the surgery itself -- even if surgery is the patient's best chance -- out of concern for their own stats?
We have an overwhelming-strong default policy of allowing and even encouraging people to collect and publish stats on their product experiences. The issue is why make an exception for this particular product - the mistake is to not even think it necessary to consider this comparison.
> If they don’t have anything better, they’ll use some pretty bad heuristics.
Will the per-hospital statistics result in any significant improvements over the heuristics? I can easily envision scenarios in which their use is mendacious or neutral (consumer ignore them just as they ignore the per-unit prices in grocery stores, but they get speciously trumpeted & misinterpreted in advertisements; consumers don't realize that one slight difference is irrelevant and flock to one hospital, driving up waiting times until $OPERATION, and people die during the interim).
More importantly for Robin's original point, is it *so* obvious that releasing the statistics would be a good thing that it constitutes outright intellectual inconsistency & cognitive dissonance?
Movie reviews are entertainment & social signaling, as you of all people should appreciate! Few seriously argue that movie critics of that ilk are doing much else, and certainly they're not predicting popularity (a few wide-spread misses like _Titanic_ torpedo that idea).
If you want to compare to movies, per-hospital stats would be ticket sales from one theater; CR stats would national box-office sales. (I've never seen any public release of the former, but the latter all the time.)
So you think no one should be allowed to publish stats on experiences with a product unless they have 100K experiences to report? That rule would put a lot of movie reviewers out of business.
Hospital ratings, though, are affected on what the surrounding populace is like, what political boundaries are near or far, what sort of reputation & services it has in attracting different patient populations, and who knows how many other factors making each hospital outcome incomparable to the next....and it still seems much easier to interpret this data than the data we currently have on hospital ratings.
People looking for good health care are going to use the best ratings available to them. If they don't have anything better, they'll use some pretty bad heuristics.
> ...I was stunned to hear a famous health economist explain it was good that the government did not disclose the med outcome data it made hospitals collect – the public might “misinterpret” outcomes, you see, not correcting right for differing patient mixes. He didn’t think it relevant that the same argument suggests Consumer Reports not publish car reliability stats, since they do not correct for driver differences.
Let me take a stab at this. CR and hospital stats are apples & oranges. You mention this, but I don't think you appreciate it.
It's been a while since I read CR, but IIRC their safety ratings, at least, were based on buying a random sample and then putting all the car models through a standardized set of automated tests (or at least professional drivers). Thus, if you see one car is safer than another, you know that the first was safer under the same sets of conditions & tests as the second; not statistically perfect, but still pretty good. Apples & apples.
Hospital ratings, though, are affected on what the surrounding populace is like, what political boundaries are near or far, what sort of reputation & services it has in attracting different patient populations, and who knows how many other factors making each hospital outcome incomparable to the next. Yes, perhaps enough sophisticated statistics can massage out all these perplexities and give a simple better-worse rating - but this is the complex interpretation that gets, say, global warming attacked, and precisely what leads to the cited dangers of interpretation.
Now, it seems to me that car safety metrics are more analogous to hospital outcomes, but maybe I'm wrong and reliability is the issue. Consumer Reports says they get survey results world-wide, and for 2008 had >100k survey responses (apparently each response covered multiple cars); this is a lot of people in a lot of circumstances, and not just new-car buyers either. What hospital statistic is this comparable to?To me, it seems more comparable to 'nation-wide death rates from disease X' than 'hospital Y death-rates from disease X'. Again, apples and oranges. Unless those proponents of restricting hospital information and releasing CR car information also oppose national statistics, I don't see any contradiction.
Personally, I would like to see medical doctors, hospitals, and all other health-care providers start posting detailed lists of the prices for their specific services, so that customers (at least, savvy ones) could compare the alternatives. After all, what other good or service sold in America do you ever first agree to purchase, and then only find out the price after you have purchased it? And yet, that's what we do with most medical care! Exactly how is that not a stupid arrangement?
i submit that the question of whether to release health care outcomes data should be not be decided on opinion or anecdote. is there any data verifying your hypothesis that making health care outcome data more accessible will improve people's decisions? if not, you are making the same error as the people who hold to keep the data secret: you're basing the conclusion on your personal intuition.
I agree that access to this data would be a good thing. However, I think your argument dismisses the data misinterpretation problem too quickly.
Here in the UK the media is constantly distorting debates about healthcare and as a result we have had to set up an independent body (NICE) to make comparisons between drugs and hospitals.
Data together with rational debate can leed to good outcomes.
Data, together with an irrational media, that falls for or exploits several cognitive biases (and errors in reasoning) in pursuing a story... doesn't leed to such great outcomes.
Access to information might have a negative effect in the short term and access to this information requires a more developed deliberative process within society.
But the right solution in my view is to have access to this information and provide better education for the public, journalists and better ways of presenting complex information. Independent oversight bodies (private or public) are a good idea too so long as they use a well thought through methodology.
Most procedures and situations are very standardized and statistics could be easily compared and understood by the general public. For example; all hospital induced infections should be public information. The rate of nosocomial infections (hospital created infections) says a lot about the sanitary conditions of the hospital and the quality of it's staff and management.
Things like outcome data for 4X heart bypasses by hospital and surgical teams can also be easily understood. You don't get a bypass unless you need it and you are in real trouble. In fact, I can't think of an area or problem where outcome data would not have been useful.
Even when asking a Dr. about outcome data, I have gotten answers like "very low risk of death" or 85% 2 yr cure rate but no references to journals or government collected statistics to back up those claims.
In a recent case (cardiac ablation), I was forced by lack of solid data to choose a Dr. on the basis of recommendations by reputation among other Dr.'s and the quality and readability of his scientific publications (which has nothing to do with how good his "hands" are).
It's not just that these stats reflect differing patient mixes, but that publicizing them would *cause* differing patient mixes. Hospitals would have a huge incentive to cherrypick their patients and refuse care to the sickest who needed it most.
My limited understanding is that this is already a problem in some areas of medicine, such as surgery. Don't surgeons routinely refuse to "accept the surgical risk" of operating on a patient who has severe health problems unrelated to the surgery itself -- even if surgery is the patient's best chance -- out of concern for their own stats?
Actually CR's reliability stats are based on owner reported problems. That is a massively self-selected sample if ever there was one.
We have an overwhelming-strong default policy of allowing and even encouraging people to collect and publish stats on their product experiences. The issue is why make an exception for this particular product - the mistake is to not even think it necessary to consider this comparison.
> If they don’t have anything better, they’ll use some pretty bad heuristics.
Will the per-hospital statistics result in any significant improvements over the heuristics? I can easily envision scenarios in which their use is mendacious or neutral (consumer ignore them just as they ignore the per-unit prices in grocery stores, but they get speciously trumpeted & misinterpreted in advertisements; consumers don't realize that one slight difference is irrelevant and flock to one hospital, driving up waiting times until $OPERATION, and people die during the interim).
More importantly for Robin's original point, is it *so* obvious that releasing the statistics would be a good thing that it constitutes outright intellectual inconsistency & cognitive dissonance?
Movie reviews are entertainment & social signaling, as you of all people should appreciate! Few seriously argue that movie critics of that ilk are doing much else, and certainly they're not predicting popularity (a few wide-spread misses like _Titanic_ torpedo that idea).
If you want to compare to movies, per-hospital stats would be ticket sales from one theater; CR stats would national box-office sales. (I've never seen any public release of the former, but the latter all the time.)
So you think no one should be allowed to publish stats on experiences with a product unless they have 100K experiences to report? That rule would put a lot of movie reviewers out of business.
Hospital ratings, though, are affected on what the surrounding populace is like, what political boundaries are near or far, what sort of reputation & services it has in attracting different patient populations, and who knows how many other factors making each hospital outcome incomparable to the next....and it still seems much easier to interpret this data than the data we currently have on hospital ratings.
People looking for good health care are going to use the best ratings available to them. If they don't have anything better, they'll use some pretty bad heuristics.
> ...I was stunned to hear a famous health economist explain it was good that the government did not disclose the med outcome data it made hospitals collect – the public might “misinterpret” outcomes, you see, not correcting right for differing patient mixes. He didn’t think it relevant that the same argument suggests Consumer Reports not publish car reliability stats, since they do not correct for driver differences.
Let me take a stab at this. CR and hospital stats are apples & oranges. You mention this, but I don't think you appreciate it.
It's been a while since I read CR, but IIRC their safety ratings, at least, were based on buying a random sample and then putting all the car models through a standardized set of automated tests (or at least professional drivers). Thus, if you see one car is safer than another, you know that the first was safer under the same sets of conditions & tests as the second; not statistically perfect, but still pretty good. Apples & apples.
Hospital ratings, though, are affected on what the surrounding populace is like, what political boundaries are near or far, what sort of reputation & services it has in attracting different patient populations, and who knows how many other factors making each hospital outcome incomparable to the next. Yes, perhaps enough sophisticated statistics can massage out all these perplexities and give a simple better-worse rating - but this is the complex interpretation that gets, say, global warming attacked, and precisely what leads to the cited dangers of interpretation.
Now, it seems to me that car safety metrics are more analogous to hospital outcomes, but maybe I'm wrong and reliability is the issue. Consumer Reports says they get survey results world-wide, and for 2008 had >100k survey responses (apparently each response covered multiple cars); this is a lot of people in a lot of circumstances, and not just new-car buyers either. What hospital statistic is this comparable to?To me, it seems more comparable to 'nation-wide death rates from disease X' than 'hospital Y death-rates from disease X'. Again, apples and oranges. Unless those proponents of restricting hospital information and releasing CR car information also oppose national statistics, I don't see any contradiction.
Personally, I would like to see medical doctors, hospitals, and all other health-care providers start posting detailed lists of the prices for their specific services, so that customers (at least, savvy ones) could compare the alternatives. After all, what other good or service sold in America do you ever first agree to purchase, and then only find out the price after you have purchased it? And yet, that's what we do with most medical care! Exactly how is that not a stupid arrangement?
Robin attacks a specific argument for the position, not the position itself.
i submit that the question of whether to release health care outcomes data should be not be decided on opinion or anecdote. is there any data verifying your hypothesis that making health care outcome data more accessible will improve people's decisions? if not, you are making the same error as the people who hold to keep the data secret: you're basing the conclusion on your personal intuition.