Beware High Standards

300,000 useless knee surgeries annually, costing $1.5B, continue in the US because a ’02 study finding them useless "was criticized as flawed."  Now a new study finds the same results but "also has weaknesses."  We sure wouldn’t want anyone to change beliefs or behavior because of a criticizable study – not when there are all those impossible-to-criticize studies out there. The news:

One of the most common surgical procedures performed in the United States — arthroscopy to treat arthritis in the knee — is useless. … [The] findings are being published in today’s issue of the New England Journal of Medicine. … The study marks the second time a major study has questioned the operations, which can cost about $5,000 and are done on hundreds of thousands of Americans each year. In 2002, a study found the operations were no better than a sham procedure for arthritis. A year later, based on that and other studies, Medicare stopped paying for the operations for severe arthritis of the knee, and the number of claims for the procedure plunged.

But the 2002 study was criticized as flawed: All the operations were done by one surgeon, and the subjects, who were all men, tended to be older than the typical patient. As a result, many surgeons continued to perform the procedures. Feagan says that as many as a third of the estimated 950,000 arthroscopic knee surgeries performed each year in the United States are probably still being done for arthritis, which affects about 27 million Americans.  …


The new study was designed to avoid the shortcomings of the earlier research; it involved men and women with an average age of 60, which is more typical. All patients received standard nonsurgical treatment, including physical therapy, painkillers such as acetaminophen and ibuprofen, glucosamine supplements and injections to lubricate the joint. Eighty-six of the patients also underwent arthroscopic surgery, which involves inserting instruments through tiny incisions to clean out any loose debris and smooth out the joint.

Tests done every six months for the next two years showed that both groups improved, reporting less pain and stiffness and more mobility. But the patients who did not have the surgery fared just as well as those who did. …

Whether the new study will settle the debate remains unclear. In an editorial accompanying the study, Robert Marx of the Hospital for Special Surgery in New York argued that the new study also has weaknesses, and that there might be patients with knee arthritis who might benefit, such as those who also have torn cartilage.  Feagan said that argument could result in continued overuse of the surgery. A second study involving 991 patients in the same issue of the journal found that many patients with arthritis in their knees also have small cartilage tears but that the tears are not the source of their pain.

Holding a high standard for what counts as a good study does not make you more evidence-based, and it is not a good thing if the factors you implicitly rely on in the absense of a "good" study (e.g., "all else equal a doc recommendation is a good sign") are even less reliable than the "bad" studies you reject. 

GD Star Rating
loading...
Tagged as:
Trackback URL:
  • kay

    The answer’s very simple: money. Doctors perform millions of operations yearly that they know are unnecessary, useless, or worse, because the people want help, and the doctors want money. If you really want to understand the primary source of knee pain, check out The Trigger Point Manuals, by Drs. Simons and Travell. All the research you could want is referenced there (warning: not for amateurs who don’t normally read medical books). They have spent their entire careers on this topic, but typical for doctors, their treatment methods are difficult and expensive. Typical for modern medical practice, most doctors know nothing about the widespread problem of trigger points; and the phrase “trigger point therapy” has been co-opted by people who use the same old failed treatments while promising something new.

    If you want to help your knee pain, get Clair Davies’ Trigger Point Therapy Workbook – it was the best $14 I ever spent. And yes, I had had MRIs on both knees, and the orthopods saw damage and recommended surgery (it’s what they do – they are surgeons, they think cutting is the answer to most problems). I had been nearly crippled for over a year, and after 2 days of this self-massage, I threw off my compression bandages for good, 5 years ago.

    It’s a mistake to believe that doctors are scientists, or that any medical practitioners have any idea of logic, or mental rigor. They might, but more likely not.

  • michael vassar

    The phenomenon whereby everything new and promising becomes a buzzword which is slapped on the same old things or on crude surface imitations of the new thing seems to be ubiquitous. What sort of institutions would prevent it? Do people want to prevent it?

  • http://drchip.wordpress.com/ retired urologist

    @kay: Doctors perform millions of operations yearly that they know are unnecessary, useless, or worse, because the people want help, and the doctors want money. It’s a mistake to believe that… any medical practitioners have any idea of logic, or mental rigor.

    The name of the blog is Overcoming Bias. I’ll need some references to evidence that doctors knowingly harm millions of people each year in order to get money. I’ll need some references to evidence to confirm that it is unwise to believe that any medical practitioners have any idea of logic or mental rigor.

    As to whether arthroscopy for arthritis is useless, apparently it is. I doubt that many practitioners were aware of that when they began doing it. Note that the finding is “arthroscopy for arthritis” is useless, not “arthroscopy has no benefit in the treatment of any abnormality of knees.”

    I had been nearly crippled for over a year, and after 2 days of this self-massage, I threw off my compression bandages for good, 5 years ago.

    This portion of your comment belongs on the previous post, “Psychic Powers”.

  • http://www.cawtech.freeserve.co.uk Alan Crowe

    Modern man is so committed to empirical knowledge, that he sets the standard for evidence higher than either side in his disputes can attain, thus suffering his disputes to be settled by philosophical arguments as to which party must be crushed under the burden of proof 🙂

  • Rob Keys

    This doesn’t show that the surgery is unnecessary, it shows that the results are comparable to a regular regimen of pain killers, physical therapy, and supplementary medications. What about the wishes of the patient? maybe they would rather have one surgery to alleviate pain in the long-term than a battery of treatments that must be repeated. also, presumably the surgery would reduce the general level of pain rather than relying on the patient to take pain-killers when pain presents itself. If you have a slow oil leak in your car you can just keep adding oil as needed or you can get the leak fixed. both solutions alleviate the problem.

  • http://yudkowsky.net/ Eliezer Yudkowsky

    Alan Crowe: Modern man is so committed to empirical knowledge, that he sets the standard for evidence higher than either side in his disputes can attain, thus suffering his disputes to be settled by philosophical arguments as to which party must be crushed under the burden of proof.

    I know you meant this to sound terribly profound, but it is.

  • kay

    Dear retired urologist: I realize I have hurt your feelings, but saying my response to the trigger point treatment is bogus, which is what I’m guessing you mean (you could have said what you meant, instead of going for a slam, you know) is betraying your bias. Have you read either of the books to which I refer ?

  • Curt Adams

    The problem isn’t high standards, it’s *asymmetrical* high standards. A highly rigorous clinical trial was required to stop doing a procedure, but not to start it. An asymmetry should go in the opposite direction, especially since bad knees can wait a few years for a study to be done. With 1.5 billion per year at stake – plus a real theoretical possibility for harming the patient (the surgery involves removing connective tissue from osteoarthritics, whose problem is primarily loss of connective tissue) – it was inexcusable for this practice to become widespread without a highly rigorous clinical trial *supporting* the procedure.

    Rob Keys: BOTH wings got medical therapy. There weren’t any patients who got surgery without aggressive medical therapy and rehab, so it’s quite speculative at best to hope that surgery alone would have most of the benefits. Certainly the data in the study would make one suspect surgery alone would have comparatively little benefit at best.

  • http://drchip.wordpress.com/ retired urologist

    @kay: I realize I have hurt your feelings.

    You seem to have an unusual gift for knowing the thoughts and motivations of people in general, doctors in particular, and me specifically. Your comments did not hurt my feelings; If they were based in fact, perhaps they would, or at least sadden me. More likely, I would accept facts as facts, and adjust accordingly.

    I am aware of trigger points, and their neurologic bases, but I have not read the books you mention, nor do I intend to; I’m rather old and I try to assign my efforts according to likely benefit in the time I have remaining. I am aware of many instances of near-miraculous recovery from all sorts of maladies, using all sorts of treatments, some of which are discussed in the blog post I mentioned. The most commonly known one is the recovery from death after three days by a man, a miracle-worker himself, who had been assigned to carry the responsibility for all the wrong-doings of mankind both before and after his lifetime. Following his recovery, he ascended into the sky and has not been seen since, although he listens to everything people say to him from somewhere afar. I’ve read that book.

  • http://www.hopeanon.typepad.com Hopefully Anonymous

    Is there a comprehensive look at economics waste due to correctable bias? What’s the order of things we should rationally be looking at? Where does useless knee surgery rank on that list? What would be at the top of that list?

  • http://shagbark.livejournal.com Phil Goetz

    I was in a doctor’s office, and overheard an orthopedist talking to a patient, telling him that he needed orthoscopic knee surgery, and that it would cost about $5000.

    The patient left, and the doctor immediately walked into another room, and told another doctor, “I’ve decided to take that vacation in Montana.”

    The other doctor asked him, “How much will that cost?

    And he answered, “About $5,000.”

    (Which caused me to wonder not only about the morality of the doctor, but also how one could spend $5,000 in Montana in one week.)

    That aside:

    Curt – Your point would be a good one, but is it true? I don’t think insurance generally covers procedures that haven’t been supported by clinical studies. But I’m not sure.

  • http://drchip.wordpress.com/ retired urologist

    @Phil Goetz

    Let me see if I understand the logistics of your story. You were in some position in an orthopedic surgeon’s office such that you could hear conversations in three different locations, that being two different examination rooms, and at least a commons room where the doctors speak to one another. If you were a patient, that would involve you being in a third exam room; if not, you were a non-doctor, non-patient allowed access to medically-privileged conversations. Picture the doors, insulation, physical distances, etc. I will assign a probability <0.1 that your story is true. If I'm wrong, then that doctor has a lot more to be concerned about than performing unnecessary surgery: he has violated a number of federal laws, the minimum fine for each patient being $10,000. He has also violated a number of ethical laws for which he could lose his state medical license. If he's the typical orthopedic surgeon, $5000 would not in any way influence whether he could take a trip. You, on the other hand, if you're lying, have only been exposed on a blog post.

    So, I've got to decide which to believe: a person who hasn't got a clue what a week-long trout fishing or whitewater trip in Montana costs, or the imaginary doctor in the story. I'll go with the doctor.

  • http://hanson.gmu.edu Robin Hanson

    Phil, most medical treatments, including most treatments covered by insurance, have no academic-quality clinical studies whatsoever.

  • http://www.iphonefreak.com frelkins

    @Phil Goetz

    Perhaps you recall the famous Business Week cover story in 2006, “Medical Guesswork?”

    “The problem is that we don’t know what we are doing,” [Dr. David Eddy] says. Even today, with a high-tech health-care system that costs the nation $2 trillion a year, there is little or no evidence that many widely used treatments and procedures actually work better than various cheaper alternatives.

    This judgment pertains to a shocking number of conditions or diseases, from cardiovascular woes to back pain to prostate cancer. During his long and controversial career proving that the practice of medicine is more guesswork than science, Eddy has repeatedly punctured cherished physician myths. He showed, for instance, that the annual chest X-ray was worthless, over the objections of doctors who made money off the regular visit.

    He proved that doctors had little clue about the success rate of procedures such as surgery for enlarged prostates. He traced one common practice — preventing women from giving birth vaginally if they had previously had a cesarean — to the recommendation of one lone doctor. Indeed, when he began taking on medicine’s sacred cows, Eddy liked to cite a figure that only 15% of what doctors did was backed by hard evidence.”

    http://www.businessweek.com/magazine/content/06_22/b3986001.htm

  • Douglas Knight

    retired urologist:
    If I’m wrong, then that doctor has a lot more to be concerned about than performing unnecessary surgery

    really? How often are these laws violated? enforced? It’s hard for me to reconcile that claim with the signs in the elevators asking doctors not to talk, but I would like to know rates.

  • http://www.hopeanon.typepad.com Hopefully Anonymous

    I think it’s a reasonable possibility both that Phil Goetz is lying and that much of conventional medical spending is ineffective signalling to show people care and to allay patient fears.

    No one else here is interested in where this is situated on an overall heirarchy of correctible economic waste due to bias?

  • http://drchip.wordpress.com/ retired urologist

    @Douglas Knight

    I’m amazed that the general public has no knowledge of the laws involving medical regulation. Did you think we were free to do whatever, like in a democracy?

  • http://zbooks.blogspot.com Zubon

    I’m amazed that the general public has no knowledge of the laws involving medical regulation.

    The general public presumably has some idea that the medical profession is regulated. That is about as specific as it gets. We might have some idea about doctor-patient confidentiality and the Hippocratic Oath. That’s about it. Really fancy folks have heard of HIPAA. I assume the average person also thinks that there are too few regulations, that doctors are robbing people blind, and that there oughta be a law against those waiting room times, although I have no polling data on that so [citation needed].

    I work with traffic records. Off the top of your head, can you think of any regulations dealing with those? Who can access them, whether companies can buy them for mailing lists, what your insurers can get, what an officer can see if you are stopped…? p>0.95 that you have been or will be in a crash. But the regulations are not that important to you unless something goes catastrophically wrong.

  • Jor

    I eagerly await Robin’s first post lamenting the fact that health economists are the ones that setup the incentive system that promotes marginally useful procedures over conservative management. So, before we just blame physcian’s for performing these procedures — remember it is economists who decided these are the most economically valuable.

  • Douglas Knight

    retired urologist,
    why should I care what the laws are, when I know that they’re continually being violated? Those very signs tell me what some of the laws are; but they also tell me that someone thinks the laws are so widely violated that the signs are useful. I’m saddened that you profess outrage when we question the sanctity of the medical profession. I can only try to eliminate all influence you’ve had on my beliefs about medicine, but I’m afraid I’m permanently tainted.