Overcoming Bias

Share this post

Randomised Controlled Trials of Parachutes

www.overcomingbias.com

Discover more from Overcoming Bias

This is a blog on why we believe and do what we do, why we pretend otherwise, how we might do better, and what our descendants might do, if they don't all die.
Over 11,000 subscribers
Continue reading
Sign in

Randomised Controlled Trials of Parachutes

Peter McCluskey
Aug 24, 2008
Share this post

Randomised Controlled Trials of Parachutes

www.overcomingbias.com
30
Share

It is tempting to react to unscientific methods of medical practice by rejecting any treatment that isn’t supported by rigorous scientific evidence.  Here’s a parody of naive implementations of evidence-based medicine that demonstrates the pitfalls of doing so:
Smith GCS, Pell JP. (2003). Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials. BMJ, 327(7429), 1459-1461.

From the paper:

Results We were unable to identify any randomised controlled trials of parachute intervention.

Conclusions As with many interventions intended to prevent ill health, the effectiveness of parachutes has not been subjected to rigorous evaluation by using randomised controlled trials. Advocates of evidence based medicine have criticised the adoption of interventions evaluated by using only observational data. We think that everyone might benefit if the most radical protagonists of evidence based medicine organised and participated in a double blind, randomised, placebo controlled, crossover trial of the parachute.

There are some interesting comments on the paper here and here.

Share this post

Randomised Controlled Trials of Parachutes

www.overcomingbias.com
30
Share
30 Comments
Share this discussion

Randomised Controlled Trials of Parachutes

www.overcomingbias.com
Overcoming Bias Commenter
May 15

Richard,Naive evidence-based-medicine does show signs of overemphasizing statistical knowledge, partly in overreaction to the use of poor causal models.Rigorously acquired statistical knowledge is generally better than causal models which exist only in the mind of doctor who has little incentive to avoid biases such as overconfidence. But "generally better" doesn't mean "always better", and the manner in which causal models are chosen can be improved.Our knowledge about the effects of hitting the ground at a particular velocity hasn't been rigorously tested. But for almost all medical treatments, the effects are much weaker, and the weaker an effect is the more rigorously we need to examine the evidence. I doubt that your causal model of how high velocity impacts cause injury is particularly well thought out. I think your confidence in your causal model is based largely on the absence of people who have any doubts about the predictions you have made so far using that causal model.

I doubt that it's important to read any particular book before reading Causality, but it does require a good deal of comfort with basic statistical theory and with the way math is taught in college level textbooks.

Expand full comment
Reply
Share
Overcoming Bias Commenter
May 15

Judea Pearl makes the distinction between statistical knowledge and causal knowledge and argues that causal knowledge is much more useful. In fact, the main purpose of statistical knowledge in Pearl's way of looking at the world is to help a person or an intelligent agent acquire causal knowledge.

The evidence-based-medicine movement shows signs of overemphasizing statistical knowledge.

The parachute example is a good illustration of the superiority of causal knowledge because the reader knows that falling out of an airplane causes injury because of the velocity with which the person hits the ground. In the vocabulary developed by Pearl and his colleagues, the velocity at which the person hits the ground screens off the effect falling out of the airplane has on the injury.

Eliezer has written that the reader should read Pearl's Probabilistic Reasoning in Intelligent Systems before attempting Pearl's Causality book, and has advised students of AI to read Tom Mitchell's Machine Learning before reading Probabilistic Reasoning in Intelligent Systems.

Expand full comment
Reply
Share
28 more comments...
Top
New
Community

No posts

Ready for more?

© 2023 Robin Hanson
Privacy ∙ Terms ∙ Collection notice
Start WritingGet the app
Substack is the home for great writing