Doctor, There are Two Kinds of “No Evidence”

I have a relative who has cancer and has been taking a particular chemotherapy drug.  It has been very successful; all of the tests and scans have been coming back negative for some time.  Recently I went along to an appointment with a fancy consulting oncologist to get his opinion about how much longer to continue with the drug.  Going into the appointment, I had the idea (based on nothing but what seemed to me like common sense) that there was a tradeoff: more chemo means a higher chance that the cancer won’t reappear, but also means a higher chance of serious side effects, and that we were going there to get his opinion on whether in this case the pros outweighed the cons or vice-versa.  What he said instead was that there was "no evidence" that additional chemo, after there are no signs of disease, did *any* additional good at all, and that the treatments therefore should have been stopped a long time ago and should certainly stop now.  I asked him what was incorrect about the (seemingly) common sense notion that additional chemo might get rid of the last little bits of cancer that are too small to show up on scans, and he said, more or less, that it’s not my idea of common sense that matters, it’s the evidence, and there is no evidence that things work that way.  So then I asked him whether by "no evidence" he meant that there have been lots of studies directly on this point which came back with the result that more chemo doesn’t help, or whether he meant that there was no evidence because there were few or no relevant studies.  If the former was true, then it’d be pretty much game over: the case for discontinuing the chemo would be overwhelming.  But if the latter was true, then things would be much hazier: in the absence of conclusive evidence one way or the other, one would have to operate in the realm of interpreting imperfect evidence; one would have to make judgments based on anecdotal evidence, by theoretical knowledge of how the body works and how cancer works, or whatever.  And good people, maybe I’m being unfair and underestimating this guy, but I swear to you that this fancy oncologist in this very prestigious institution didn’t seem to understand the difference between these two types of "no evidence."  So while he had a very strong and very (generally) laudable instinct that one ought to base one’s medical opinions on evidence rather than instinct, he seemed to be unable to avoid what strikes me as a pretty fundamental mistake.*  I’d love to hear thoughts about this, particularly from doctors who either have something to say about whether this is a common mistake among doctors or who have something to say about the chemotherapy question itself.

*The most generous possible interpretation of what went on, but which would require me to attribute to him a thought process that he did not express at all, is that he understands the difference between the two types of "no evidence" but has come to believe that doctors’ interpretations of imperfect evidence will systematically lead them to over-treat and so has adopted a rule of "do nothing unless there is strong evidence that you should do something" as a second-best optimum.

GD Star Rating
Tagged as: ,
Trackback URL:
  • Douglas Knight

    My first thought was that the phrase “null hypothesis” would have made communication easier, but then I wondered if that would merely have made communication of “the doctor is not an idiot” easier without making communication of the truth easier.

  • David J. Balan, you would be pretty surprised at what qualifies as a “top x% doctor”, and I suspect this kind of (inexcusable) confusion among doctors is pretty common. I’ve seen very few doctors who could handle such a distinction. One recent experience I’ve had is in trying to treat my back pain, which is pretty bad, and started a young age, and doesn’t respond to pretty much any treatment method (5 physical therapists, 2 chiropractors, ~8 low-grade medicines, acupuncture, electrotherapy, massage therapy, steroidal injection, and others I probably can’t remember).

    To get to the point, a back surgeon recommended trying another steroidal injection (which another doctor would administer). I asked, “So, I should probably need to send over the records from the previous, so the doctor can know what methods didn’t work.” The back surgeon responded, “No, that probably won’t be necessary.” “Why not?” “Well, each doctor has his own method for determining what to do.”

    Red flag.

    “But, this is all based on some rigorous procedure for determining what to do, which is based on a statistical analysis of all past steroidal injection methods, right?”

    Um, no, it’s not like that.

    “Is it more methodical than rolling dice to decide what to do?”

    *smirk* Heh, well, each doctor has has own thing. Here, give this form to the girls up front and they’ll set you up.


    And this doctor, like in your case, was overflowing with awards.

  • J Thomas

    I am not an MD. See if the following reasoning makes sense to you:

    A new chemotherapy drug needs rigorous testing to show that it does something that might be useful. If people who take the drug tend to get remissions when people who had that sort of cancer before the drug was available tended not to, that looks pretty good.

    After you get a remission, how much longer is good to take the drug? You won’t know until you establish a baseline. So if you stop treatment when you think you have a remission, then you can find out how many subjects get symptoms again. That establishes the maximum benefit from extended treatment. Now, you might get faster results to give treatment for random times and see which length of treatment gets the best survival. Maybe too much treatment results in disability from the chemotherapy while too little results in disability from a recurrent cancer. But if you put the maximum number of people on the minimal treatment schedule then that’s the quickest way to find out whether there’s anything there to study. If very few of them relapse then further treatment is definitely not worth doing, unless you can find out which ones will do that.

    So if you’re pretty sure that lots of cancers will recur, then it makes sense to go ahead and try to find out which treatment is best. But if there’s a strong chance that it won’t be an issue then better to just keep records to find the new baseline in case you’re wrong.

  • Ian C.

    I’d have to side with the doctor on this one. You don’t need statistical evidence to know that if there’s no disease you’re wasting your time applying treatment. By what causal mechanism is nothing effected by something?

  • James Andrix

    Given that he’s in the business of giving poison to people when he knows he has to, your generous interpretation might well have been adopted unconsciously, and could be very hard to articulate.

    After ‘Do no harm’ is pounded into you, poisoning a patient on a hunch might seem very alien. (for better or worse)

  • @ Ian: you don’t “know that […] there’s no disease”. What you know is only that “tests and scans have been coming back negative for some time”, which is not at all the same thing. Tests and scans have limited resolution, and from the biology of cancer it is well known that even a small colony (if it exists), well below the scan threshold, can eventually grow back into a new tumor.

  • Thanatos Savehn

    The problem here is that you, like most people, believe that cancer can be cured. Certainly there are people who go into extended remissions and then manage ulitmately to die from something else. But save for a very few cancers they aren’t and never have been cured.

    If you’ve inherited a mutation in a cancer suppresion gene like p53 it’s just a matter of time. If you’ve overcome one sort of cancer then if you live long enough you’ll eventually roll a 0-0-0-0-0-0-0-0-0-1 on the 10 twenty siders you’re rolling millions of times a day and the Stochastic God will smite you yet again.

    After all rounds set out by whatever protocol has been found to be best (and by the way, at MD Anderson here in Houston they see lots of outcomes) you’re either cancer-free or the cancer is unreachable by the treatment. Thereafter, any further treatment is merely needless torment and those who demand it are suffering from the risk/benefit miscalculation described by Gilovich et al in “Heuristics and Biases”.

    Note: the referenced heuristic has one great value – it makes certain doctors and hospitals beaucoup bucks.

  • retired urologist

    Chemotherapy is not a treatment designed to “cure” cancer, i.e. eliminate all the injuriously mutated cells. It is designed to reduce the abnormal cell load to a level low enough for the immune system to eliminate. This is tricky, or even paradoxical, since the immune system failed to eliminate it when the mutation first occurred. “Cancer” cells share most of their mechanisms with “normal” cells; the drugs are designed to attack mechanisms that are far more common in the abnormal cells than the normal ones, but there are no drugs that can do this exclusively. Some mutations are exquisitely sensitive to combinations of drugs that address more than one mechanism of abnormality, e.g., the choriocarcinoma that Lance Armstrong has survived. There were no survivors known for this mutation when I was in medical school, whereas now the “cure” rate is around 95%. But if he is “cured”, the final part of the job was done by his immune system. Radiation, on the other hand, kills indiscriminately. Surgery aims to eliminate the “nest”, hoping there are no other sites of contamination.

    The doctor in this anecdote was saying, in the typical “I know more about it than you do” fashion that we often see in professors (?of any subject) that the abnormal cell load goal had been reached; at this level, a higher and higher number of normal cells are killed for each abnormal one. To kill the last abnormal cell, virtually all of the normal cells would die as well, along with the patient. If the immune system cannot handle the goal load of abnormal cells, there will be no success in the long run. Doctors don’t like to say this to patients. In the end, the “cures” are selections of those with the best immune systems, for which, so far, there is no accurate test.

    As to the two types of evidence, he should (and I’m certain he had the knowledge to do so) have answered that your first scenario is correct: his field is replete with a plethora of studies about reducing abnormal cell loads to the magic level, rather than eliminating all mutations. In fact, those studies are the entire basis for the field of medical oncology. We all know of well-informed, highly intelligent people who represent the correct position on a topic, but whose arrogance interferes with their persuasive skills.

  • Retired Urologist, you should definitely be blogging. I wouldn’t be surprised if it ends up one of the top medical blogs on the internet. And your posts have the makings of a really, really good book.

  • Anonymous of /jp/

    Seconded. That was a very clear and interesting reply.

  • David J. Balan

    Retired Urologist,

    Thank you *very* much for your thoughtful reply. But I want to clarify one thing. I now understand that it is the body, not the chemo, that gets rid of the very last little bits of cancer. That is not, however, exactly the same statement as saying that chemo can do no additional good once the tests and scans have come back clean: it seems theoretically possible that there could still be some bits of cancer that are small enough not to show up on scans but that chemo can still be useful in eliminating, even if they are not the very last bits. Are you saying that this is false as a matter of fact, and that it is an established result that once you have clean scans the “magic level” has already been reached and chemo can do no additional good?

    Thanks again!

  • retired urologist

    @Balan: Are you saying that this is false as a matter of fact, and that it is an established result that once you have clean scans the “magic level” has already been reached and chemo can do no additional good?

    No. I am saying that if our current methods cannot detect any other cancer cells, there is no defensible reason for further use of a harmful therapy (notice I don’t say “potentially harmful”: chemo kills normal cells). Suppose your relative had never had a cancer diagnosed, but asked his doctor to give him prophylactic chemotherapy anyway, in case there were some undetected cancer cells in his body. In reality, your relative’s concern would be legitimate: each of us apparently has cellular mutations daily that would become “cancers” were it not for our immune systems. I know of no doctor who would agree to the request, and I know thousands (?millions) of personal injury attorneys who are hoping that he would.

    From the viewpoint of a strict, double-blind controlled experiment, if cancer patients whose abnormal-cell load had reached the undetectable level were randomized to groups that received more chemo, received a similar placebo, or received nothing, and when the code was broken, we found that those in the first group had a l longer remission, we still would not know whether it was down to the chemo killing cancer cells, because we had no evidence of cancer cells to begin with. Maybe the chemo has some other previously unknown effect. For certain, the number of non-recurrences in the no-treatment group would also be present in the chemo group, but some of those would die from the chemo (for sure).

    You might ask: “what difference does it make that we don’t know why the extra-chemo group has more remissions, so long as they did?” If chemo were safe, it wouldn’t. But chemo attacks the immune cells as well. People with a diminished immune system have 10 times the incidence of cancer as the general population (including organ transplant patients on immunosuppressive drugs). For those who took more chemo, and eventually had another cancer, we could not rule out the chemo’s effect on the immune system as part of the cause.

    Finally, from an American societal viewpoint, I’m not sure the average non-doctor realizes the influence physicians give to the threat of medical malpractice litigation. This is the only nation in the world (I think) that allows contingency fees in litigation. It is a lottery ticket, especially for the family of a patient who dies while being treated for an undetectable disorder. That’s not the same as “evidence”, but it’s a fact.

  • Jeremy

    Retired Urologist: “Radiation, on the other hand, kills indiscriminately.”

    Is it not the case that radiation has a much stronger effect on more rapidly dividing cells? I believe because there is less time for radiation-damaged DNA to be repaired before cell division.

  • retired urologist

    @Jeremy: Is it not the case that radiation has a much stronger effect on more rapidly dividing cells? I believe because there is less time for radiation-damaged DNA to be repaired before cell division.

    Yes, it is. The indiscriminate part is that is KEEPS killing, c.f., the radiation-related malignancies emerging many years later from Hiroshima, Nagasaki, and Chernobyl. Ironically, I don’t recall seeing any comments on this aspect of nuclear weaponry on the post Hiroshima Day. Modern radiation therapy uses rotational, computer-guided beam mechanics and computer-guided placement of interstitial radioactive seeds to focus on the exact desired site, and doses are carefully calibrated, all in distinction to the events above. Yet one sees skin breakdown, colon erosion, etc. at the site of radiation Rx twenty+ years later. That what I meant by indiscriminate. Perhaps I should have added “and progressive”. I did not mean to imply “not good”.

  • There’s No Evidence Like No Evidence

    Here’s an especially wise observation by David Balan:So then I asked him whether by “no evidence” he meant that there…

  • Constant

    No. I am saying that if our current methods cannot detect any other cancer cells, there is no defensible reason for further use of a harmful therapy…Suppose your relative had never had a cancer diagnosed….

    There is a difference between a patient who recently had detectable cancer and who no longer has detectable cancer, and a patient who never had detectable cancer and still has no detectable cancer. While the patients are indistinguishable by current tests (in that neither has cancer detectable by those tests), the current test result is not the only knowledge that we have available – we also have available knowledge of their recent histories, which are different.

    if cancer patients whose abnormal-cell load had reached the undetectable level were randomized … we still would not know whether it was down to the chemo killing cancer cells

    But you could also compare (a) patients with currently-undetectable levels who recently had detectable levels with (b) test subjects with currently-undetectable levels who never had detectable levels. There might be a difference between how these groups respond to chemo, placebo, and nothing. If there is, then the inadvisability of giving chemo to people with no history of detectable cancer does not necessarily extend to an inadvisability of giving chemo to people with recently-but-not-currently detectable cancer (i.e. for some period after the cancer has ceased to be detectable).

  • retired urologist

    @ Constant: test subjects with currently-undetectable levels who never had detectable levels … how these groups respond to chemo

    Test subjects that have never at any time had detectable levels of cancer fall into a category that we label “healthy”. As I understand it, you want to recruit healthy people and give them chemo? Please give a brief example of the proposal you would submit to the FDA requesting approval of your study. Also, please transfer all your assets to an offshore bank that does have an extradition agreement with the USA, if you plan to keep any assets.

    As for the cancer patients, what would be your end-point for the chemo in patients who formerly tested positive and now test negative? It can’t be any kind of test, because there’s no better result than “normal”, and they already have that.

  • retired urologist

    retired: an offshore bank that does have an extradition agreement with the USA

    CORRECTION: should be “does not have …”

  • constant

    retired: you’re not taking the hypothetical experiment in the spirit that it is offered. It’s hypothetical.

    The endpoints I have in mind are time to relapse and time to death.

  • statistician

    retired urologist seems to be making a standard error in the double blind literature. Elementary as it sounds, absence of an effect in a double-blind experiment is not the same as evidence of absence. Most experiments and statistical protocols are designed to minimize type 1 error. Medical research is not very good at discussing type 2 error. It may be sensible to assume that studies showing no net effect as dispositive with respect to discontinuing chemo, but it really depends on how strong the side effects are. Absent direct evidence about the possibilities of undetectable cancers and the net costs of further radiation (of the sort mentioned in Constant’s hypothetical comparison that wouldn’t be funded by the FDA) it is accurate to say that the doctors DON’T have strong evidence that continuing chemo does no good. The reasoning may favor their conclusion but that is precisely the sort of missing evidence that this post seems to be pointing to.

  • retired urologist

    @statistician: your entire post

    This post is typical of the anonymous internet blog, of which admittedly I am a part. It is ridiculous, in that it has no relationship to the real world. First, technically, no mention of continuing radiation has been made (I assume either alcohol or drugs may be a factor in the comment content). Second, the FDA does not fund any type of medical therapy, as anyone in the medical field would know. Third, I am CERTAIN that neither Constant nor Statistician has ever had to administer chemotherapy nor see the direct effects it has on the lifestyle of the patient. To give it for no reason is ethically unconscionable. Fourth, doctors do not treat hypothets, unless they present living and breathing in the office.

    I will refrain from further comment on this topic unless the original poster has a point.

  • David J. Balan

    Douglas Knight, In the type of Bayesian analysis that is always getting talked about on this blog, one doesn’t specify a “null hypothesis.” Rather, one starts with “prior” beliefs and with some notion of how confident one is in those beliefs, and then updates those beliefs based on new evidence. The problem with specifying a null hypothesis is that in doing so one puts a pretty big thumb on the scale in favor of that hypothesis. There might be occasions when this is a good idea, like in a criminal trial where it is a good idea to have a null hypothesis that the defendant is not guilty, but specifying a null hypothesis probably wouldn’t have been the right thing to do here.

    J Thomas, I don’t have a good sense of the optimal way to carry out a research program for definitively figuring out the answer to this question. I just want to know if somebody had done such a research program (optimally or otherwise) or not.

    James Andrix, You might be right, and if so this would be a defensible, though not obviously correct, position for the doctor to take.

    Don Geddes, I agree.

    Thanatos Savehn, the question is precisely whether or not a particular “protocol has been found to be best.”

    Retired, I think you’ve done a bit of an injustice to constant and statistician. They have both pointed out what seem to me plausible (which very very far from the same thing as correct, and the actual correct answer is what I’m trying to find out) scenarios in which there could be some cancer-fighting power in additional chemo beyond the point where the scans come back clean, and yet this fact could be not currently known to medical science. That’s not to say that more chemo is the answer (again that’s what I’m trying to figure out), as we certainly have no strong basis for saying that such cancer-fighting power exists AND that it justifies the (potentially large) costs of further chemo. You seem to have started out saying that definitive studies on this point do exist, but your later comments seem to say that they don’t. You can be very sure that I’m not trying to score points here. I really, really want to know the best answer to this question.

  • constant

    Here’s something that seems to raise the same issue:

    A closer look at the Norton Simon model (incorporating the Gompertzian kinetics and the log-kill hypothesis) reveals that even after a complete response (CR) is achieved clinically with combination chemotherapy, there exists substantial subclinical disease burden that can cause a relapse. This subclinical disease can possibly be eradicated by another two log kills, prompting the use of two more cycles of chemotherapy after CR as consolidation.

    To clarify what is being said, complete response means:

    The disappearance of all signs of cancer in response to treatment. This does not always mean the cancer has been cured. Also called complete remission.

    Complete response is absence of evidence of disease. The definition explicitly points out that absence of evidence is not proof of absence. The above quote from the letter to the Journal of Clinical Oncology points out that it may be beneficial to continue chemotherapy even after the patient has achieved a complete response – that is to say, after the tests have all come back negative.

    The terms “consolidation chemotherapy” and “complete response”, when googled together, seem to be fruitful in finding similar material.

  • Douglas Knight

    Yes, I know that hypothesis testing is evil and will lose you points on this blog, but you’re the one who admitted to trying to talk to an outsider!

    The original problem was communicating with the doctor. I think the phrase “null hypothesis” would have helped you distinguish the two notions of “no evidence” to him. It’s not a very precise belief; it’s not like I have a hypothetical dialog in mind. But your response seems like such a dramatic non sequitur, I thought I’d try again.

  • To kill the last abnormal cell, virtually all of the normal cells would die as well, along with the patient. If the immune system cannot handle the goal load of abnormal cells, there will be no success in the long run.

    I think this is both true and irrelevant to the question of target duration of therapy. When drug therapy is used in the adjuvant setting (e.g., secondary to the resection of the primary tumor) it is often the case that there is no detectable disease remaining. One gives drug therapy with the idea of destroying undetected disease. Whether the specific drug therapy is a chemotherapy, an immunotherapy, or a specific antibody to some cell-surface marker overexpresed by cancer cells is irrelevant. In all cases, optimal duration of therapy is determined empirically. For example, the drug Herceptin is currently being evaluated in the HERA-2 study, which tests a one year course of Herceptin vs. a two year course of Herceptin. In this study, patient’s primary tumors have been excised and they retain no detectable disease. Thus, retired urologist’s statement:

    I am saying that if our current methods cannot detect any other cancer cells, there is no defensible reason for further use of a harmful therapy

    is not strictly accurate. This is simply what adjuvant therapy is. You are treating people with no detectable disease with the aim of preventing recurrence. Very toxic therapies are used in this setting all the time. As a practical matter, duration of therapy has been limited by therapeutic index. But as we develop therapies which, although potentially harmful, are not *as* harmful as chemo, it may make sense to extend the duration of adjuvant therapy, particularly in patients with high risk of recurrence (which does not necessarily correlate with the presence of detectable cancer cells).

    (note: I am not an MD, but do work in drug development)

  • retired urologist

    *First*, my apologies to Constant and Statistician, as suggested by David Balan. Their comments, especially those of Constant, have merit with regard to the question asked: are there conclusive studies that show chemotherapy beyond the point of detectable disease to be of no benefit? I allowed myself to speak from the point of view of the real practice of medicine rather than stay on topic, and I criticized those who did. My bad. I will try to do better in the future, as I sincerely enjoy participating with this group of obviously intelligent and informed people.

    For the most part, the *practice* of medicine does not work like a science. It is extremely regulated, and severely policed, at several levels. I can assure you that the “fancy consulting oncologist” was aware not only of the studies that Constant has cited, but of all the studies ever published on the subject, many of which you could not read without being a subscriber to the journals, or going to a library that subscribes. He also was aware of his limitations in choosing chemotherapy regimens. While in the USA doctors may *legally* use any FDA-approved drug for any purpose they see fit (e.g., anti-hypertensives can be prescribed for growing hair), such use does not eliminate the civil liability, nor the professional risk of loss of license (regulated by one’s peer doctors). In the case of dangerous drugs such as chemotherapy agents, there can even be criminal liability if the doctor is deemed to have caused loss of life by going outside proven guidelines. At least in the USA, you will not see an oncologist adding extra agents to a protocol or going beyond the goals of a protocol unless the patient is enrolled in a federally-approved human study group. I do not know the cancer-type of Balan’s relative, and if I did, I surely would not be aware of the current chemo protocols; the oncologist did.

    The method by which one could get the desired info from the doctor is this: once he had said, “there was “no evidence” that additional chemo, after there are no signs of disease, did *any* additional good at all, and that the treatments therefore should have been stopped a long time ago and should certainly stop now,” the question would have been: “Are there any human-study protocols in progress investigating the use of chemo for this type of cancer beyond the point of undetectability, for which my relative could apply for enrollment?” I suspect that the oncologist was discussing the *particular* cancer in question, rather than cancers in general. In fact, I’d bet on it. The evidence that extra chemo beyond remission helps the outcome of Hodgins Disease (Constant’s citation) is applicable to Balan’s situation only if his relative has Hodgkins Disease, and is deemed to fall into the same group as discussed.

    Lastly, unrelated to the discussion of evidence, but related to patient-doctor communication, those who have survived medical specialty training tend to be very defensive about justifying their opinions to those who are not in the fraternity. The many years of sleep deprivation, intellectual abuse, and stress related to providing services affecting human outcomes when one is still a novice tend to cause an attitude, whether or not expressed, of: “We’ll discuss this on an equal level once you have your union card.” It is similar to a career Marine being questioned about his battle plan by a lifelong civilian. I *realize* it is not right; I’m just saying.

  • David J. Balan

    Retired, Thanks for the thoughtful new comments. I hope you don’t think I’m being pedantic, but I still don’t feel like I’ve got a final resolution on this question. Constant seems to have found a citation (with which you have not found any fault) that extra chemo beyond the point of clean scans *does* help for at least some patients with Hodgkins disease. My relative does not have Hodgkins, so this is not directly on point, but it does seem to suggest that the “common sense” intuition that extra chemo can meaningfully fight cancers that are present but too small to detect has some merit. Ben A. has made a similar point. Maybe someone has done a similar study for my relative’s cancer and conclusively found the opposite effect, but that’s not what the doctor said. So as I sit here right now, I still don’t know what the real answer is to this question (if it helps, the cancer is endometrial cancer) and the doctor doesn’t seem to know either.

    Note that I am not at all saying that I think the right answer is more chemo. It’s totally possible to conclude that there may be some additional benefit to more chemo but that beyond the point we’re at now (clean scans for a long time) those benefits do not justify the costs. But I’m really concerned (both for my relative and for humanity) that it seems like this decision is being made not based on a full weighing of the costs and benefits, but on an analysis that seems to refuse to consider, for no good reason that I can see, one possibly relevant factor.

  • retired urologist

    @D J Balan: intuition that extra chemo can meaningfully fight cancers that are present but too small to detect has some merit.

    Indeed it does, and it has been proven to give statistically significant benefit in some cancers, when carefully balanced against toxicity. Endometrial is not one of them. I have spent the morning reviewing aspects of endometrial carcinoma and it’s treatment. To address your specific question directly (which I should have researched properly for the first comment I made), there has never been a human, controlled study addressing the issue of further chemo in women with advanced endometrial carcinoma who have experienced a complete response (CR) to chemo. Chemo is not normally recommended in the earlier stages of the disease authoritative current summary so I assume your relative had metastatic disease. The studies that have been done, and that have achieved CR’s, used either fixed protocols for numbers of doses, or weekly protocols that continued until either CR or toxicity weekly until response. In this study, which is recent, the opening line is: “There is no generally accepted standard chemotherapy in treatment of advanced and recurrent endometrial carcinoma,” so that gives a clue as to where we stand with the issue.

    I reviewed about 200 studies in all; these are the most relevant, authoritative, and informative. I cannot make a link to the summary list of the others because it requires my professional subscription info for access.

    Your relative has achieved an outstanding, and unusual, response. I wish her continued good health. If you would like for me to research some other aspect, please ask.

  • Another somewhat sceptical statistician

    DJ Balin, I wrote this post because you seem to be still concerned. As I prepare to post it, I see retired urologist has posted again with much solid information, so perhaps it is unnecessary. I offer it specifically to address your concerns about the level of the evidence.

    First of all, it is your relative’s body and mind and dollars that are involved. If he/she feels it is worth it in terms of cost and pain to take additional radiation, chemo or whatever, then go for it.

    But I do not think there is any rational, objective reason for recommending this. I will explain why I think so:

    By the square root rule (which you probably know since you mention type 1 and type 2 error), it takes a study of 100 patients to get evidence good to 10%. It takes 10,000 patients to get evidence good to 1% uncertainty. Pont one percent (.1%) takes a study of one million patients. And so on. Point zero one percent would require a study of one hundred million patients. Clearly, at some point, the evidence runs out short of absolute certainty. It *could* be true for some small percent of patients that additional therapy does good, say .1%, but it either does nothing for or actually harms the patient in the other 99.9% of cases.
    However, if this number was as high as even 3%, I am sure we would know about it. You can adjust this number up or down based on your degree of confidence in our medical system. But assuming I am correct in judging that this is a pretty small number, then it comes down to your relative’s judgement of the value of a possible benefit with very small probability (and for which there is no reliable evidence) versus the cost in dollars and pain of continuing treatment. ((Totally irrelevant, but coloring my evaluation, a relative of mine recently underwent cancer treatment. The cost in dollars was very significant, but the cost in pain of both the radiation and chemo was even more so. He/she wanted the pain to stop as soon as possible, consistent with preventing recurrence.))

    So the issue of possible benefit comes down to projecting the endpoint beyond the level of precision of the available statistical evidence. Do you think a simple linear projection is close to right, or do you think the curve turns around at small numbers? And do you think your relative falls into that very small subset? (For instance, there is strong evidence, though still very controversial, that a little bit of radiation is good for you.)
    Given the variability of all the factors involved, I suspect there is always a very small minority that differs from the majority, i.e. that there is a small fraction of patients who would benefit from continued treatment. But I also suspect it is vanishingly small and consider seriously that it could be zero. But I guess (and this is strictly a guess, based on the square root rule cited above) that the current noise level is probably some where between 1 and .1%. So there can be no credible evidence beyond the statistical noise level, whatever it is.

    Also, there is always a trade off. Many drugs are widely prescribed because they benefit 95% of patients, even though 3-5% have serious adverse side effects. But you should choose if you take the risk or not.

    Finally, your particular doctor may or may not be aware of type 1 and type 2 error etc. but the entire medical establishment is not that stupid. So don’t worry about this point.
    Going on, medical statistics are far from perfect, and could be much further improved in this computer age (with some potential loss of privacy and freedom), but they are pretty reliable at the present time.

    Therefore my opinion is: your relative’s choice on continuing treatment, but no statistically supported reason to doubt your doctor’s advice (to discontinue) beyond the noise level.

  • Alan

    @ retired urologist. Welcome to the blog. Not only do your posts display an in-depth understanding of the medical issues under discussion, you also demonstrate an uncommon degree of compassion, sensitivity and concern. One imagines that your former patients experienced regret in your retirement.

  • Another somewhat sceptical statistician

    One more point: the placebo effect. There is very strong statistical evidence that placebos “work”, they just don’t work the way the patient thinks they work. So if your relative thinks that the additional treatment is working or needed, that is an important fact to be considered.

  • David J. Balan

    Retired, Thank you *so* much for taking so much time in researching this question. I am blown away. Alan is right that your former patients are missing out, and HA is right that you have a special voice and you should find a way to get it out into the universe.

    You are right that my relative did have metastatic disease, and you are also right that by all accounts she has had a very unusually good outcome, far better than I would have dared expect a few years ago.

    As for the substance, here is my current best understanding of what’s going on: (i) there *is* in fact some at least some merit to the general “common sense” notion that additional chemo beyond clean scans can have additional cancer fighting effects; (ii) there is no evidence of this for endometrial cancer, but mostly because no one has really looked–the second kind of “no evidence”; and (iii) despite this, the right answer, given that the scans have been coming back clean for a while and all other relevant considerations, is still probably in favor of discontinuing. Is that right?

    Thank you again. You have been enormously helpful.

  • D J Balan: Thank you *so* much for taking so much time in researching this question.

    You are quite welcome. I’m very sorry for the situation that made it necessary.

    @you have a special voice and you should find a way to get it out into the universe.

    Thank you for such kind remarks. I have two ex-wives who might wish to debate you. I have a lot of spare time on my hands; following HA’s recommendation, my new Blog is here. If any readers would like to have a discussion, or educate me on any aspect of transhumanism or extended life, or request an opinion about medical *issues* (not recommendations for medical therapy or suggestions of diagnoses), please make a comment on my blog, which currently contains only the inaugural post.

    @ Is that right?

    I *think* that it is, but you have educated me about thinking as much as I have educated you about medical issues. As to *right*, if you know what that is, tell Mr. Yudkowsky so that he can incorporate it into his fAI and get it built. If there really is such a thing as “time”, mine is running out.

    Best wishes.

  • Thanatos Savehn

    I write again, this time to object to the “bits of cancer” nonsense set out by some above.

    One of the best works on cancer is DeVita’s “Cancer: Principles & Practice of Oncology”. Chapter 17, 6th Ed. p. 291 addresses the issue. Under “Adjuvant Chemotherapy” the authors write:

    “There was initially great excitement with the concept of using chemotherapy as an adjunct to local treatment. The rationale for adjuvant chemotherapy was to treat micrometastic disease at a time when tumor bulk would be at a minimum, thereby enhancing the potential efficacy of drug treatment. It was assumed that drug therapy, at this stage, would result in a much higher cure rate.”

    You can guess the current status of the story by the use of the past tense in the preceding paragraph.

    Here is a tiny part of the problem. p53 is responsible, at least in part, for causing defective cells to self destruct. It turns out that like little M-Class starfreighter Nostromos the cells in your body each have self-destruct mechanisms that are executed (apoptosis) when the proper command signal (cytokine key) is received from Warrant Officer Ripley (your immune system). But what if the system loses the ability to receive or understand the command? And what if that system isn’t just a stupid computer but instead is a pluripotent stem cell (queen) that is churning out huge numbers of immature, useless (to you) and ultimately destructive (i.e. cancerous) daughter cells?

    And what if those daughter cells are susceptible to chemo but the queen is not?

    In that case the chemo (which is designed to destroy the bulk of the tumor) will set the clock back to a pre-clinical point by debulking the tumor but not killing the queen. For a slow-growing cancer that takes decades to manifest you’ve rolled back the odometer and may have another 40 years; for an aggressive one you’ve got months.

    So you say “hey, but I’ll keep the cancer at bay by taking chemo as a prophylactic”. Evolution answers that question. The malignant queen isn’t happy to just churn out identical copies of partially differentiated daughters who get blasted to pieces by Ripley. Oh no. She continues accumulate mutations as she rapidly produces offspring; and from time to time she produces sisters whom she sends out into the unhappy world to do the same.

    Eventually one of those mutations leads to offspring immune to the chemo. That’s why first remissions are generally followed by much shorter second remissions (if at all) which are followed by salvage regimins (experimental Hail Marys) which are followed by palliative care. (Hairy cell leukemia being an exception).

    So what then is the point of using a weapon to which both the queen and her brood are immune? Especially if that weapon damages the user and is itself a potent carcinogen quite capable of inducing a high grade cancer like acute myelogenous leukemia in the case of alkylating chemo drugs?

    I understand that there are healthy women having mastectomies and hysterectomies because they are more afraid of dying of say uterine cancer (low risk) than they are of dying of diabetes – after multiple amputations – (high risk). And I’m ok with grown ups doing whatever they want with their own bodies. I’m just pointing out the bias at work here described by others that make some people take risks that are, from a Bayesian perspective, foolish.

  • retired urologist

    For the few who still may be interested in this thread (and like David Balin, may one day have need for unbiased thinking about cancer treatment):

    @T Savehn: You can guess the current status of the story by the use of the past tense in the preceding paragraph.

    DeVita’s 6th edition on Oncology was published in 2001. It is a volume with multiple authors, each of whom was assigned his/her chapter(-s) about three years prior to publication. Most of what they write that is not based on personal studies is from citations that are 3 to 10+ years old at the time, plus the 3 years it takes to compile and publish the edition. For instance, there could be no knowledge in that tome of the long-term outcome of Lance Armstrong (treatment 1996) and others like him with non-seminoma germ cell testicular cancer. Check the 8th edition. In a dynamic field like chemotherapy trials, all current info is from researcher-to-researcher communication, and presentation of current papers. Text books are always obsolete at the time of publication.

    @T Savehn: (I) object to the “bits of cancer” nonsense set out by some above.

    The problem here is discussing mutations from various cell lines as if they were the same actors. Seminomatous testicular cancers are exquiisitely sensitive to radiation, while radiation has no beneficial effect on testicular choriocarcinoma at all; yet both may arise from the same testicle. Consequently, one observer might say that radiation is a great way to treat testicular cancer, while another might say that it is a terrible treatment; both would be correct, just as both would be incorrect. In the original post, the cell-type of the cancer in question was not revealed; the discussion became more meaningful once the discussants knew that it was endometrial carcinoma, for which there is no known benefit of chemotherapy beyond normalization of tests (complete response).

    Lastly, Balin’s actual point was that there is a major difference between the position, “There is no evidence that further chemo helps, but that is because we haven’t run the clinical trials yet”, and the position, “There is no evidence that further chemo helps, because double-blind propsective studies have shown this result.”

  • randy

    i am biased against doctors. let me put that on the table right away.

    i think that there was a time when doctors, especially those working in prestigious places, were amongst the top 1% of 1% of intelligent people. i see an overall slippage of education, and all sorts of perverse motivations have cropped up due to economic factors.

    most doctors are people who want three things 1) money, 2) prestige, 3) a feeling of being altruistic. they are willing to put up with a tremendous amount of “work” and dedicate years of their lives to pursuit of their MD, but that education has mainly deteriorated into a signalling mechanism.

    so i am not surprised to hear you relate this story. maybe your generous interpretation is the right one, but i feel like it is not.

  • retired urologist

    randy: here are some references and comments, titled “Doctor, are you FDA-approved?”, about your exact topic. I agree in a general sense with what you are saying, but I doubt there is any unbiased evidence that doctors were ever “amongst the top 1% of 1% of intelligent people”, that there is “an overall slippage of education”, nor that “most doctors” are (fill in the blank). Overall, it would be economically unwise for doctors to try to kill their patients, but the HMO concept has shown that incentives to keep patients healthy don’t work, either. As to an MD degree being a “signaling mechanism”, that is not the fault of the doctor, but rather of a biased, uninformed, and basically dumb American public.

    What would be the characteristics of the individuals in a career entity that you admire?

  • There’s a recently released book that some of the posters to this blog might find interesting. I just read Jerome Groopman’s book, How Doctors Think. Groopman describes some of the common thinking errors that physicians (like all of us) are prone to, using some stunning case histories to illustrate his points. I suspect that there’s a lot of variation in how much training in research design and statistical analysis that physicians receive, which would affect how sophisticated their understanding of basic ideas (such as the limits of correlational research to demonstrate causation) is. What Groopman emphasizes is that doctors are taught nothing about the cognitive errors that contribute to medical misjudgments and misdiagnoses. In his book, Groopman laments the fact that doctors are not routinely taught about cognitive errors and ways to dminish them as part of their medical training.

    Groopman discusses the role of “managed care,” and the use of decision-trees, etc. as additional contributors to a lack of critical thinking by physicians. He also suggests what kinds of questions patients can raise to essentially help their doctors avoid some of those errors. Although these questions won’t “cure” the problem, they are a beginning – and Groopman urges patients who don’t receive satisfying answers from their doctors to seek another physician.

  • Carol

    OK. So now it’s almost 10 months since this thread ended. But I am left with the question regarding Hodgkin’s Lymphoma – where are the studies that show that it is benefical to continue adjuvant chemotherapy after evidence of no disease (per CT/PET Scans and Bone Marrow Bioposy)? It seems the the oncologist standarly promotes two cycles (total of 4 sessions) past no evidence. If one wanted an “insurance policy” what is to say that one cycle is not enough? Are there any studies that say that other consolidation therapies (perhaps of the alternative variety) along with therapy to boost the immune system are valid; or not?

  • Pingback: On Ignoring Good Ideas (in science!) — Part 5 – Bartleby's Backpack()