Discover more from Overcoming Bias
This is a blog on why we believe and do what we do, why we pretend otherwise, how we might do better, and what our descendants might do, if they don't all die.
Over 11,000 subscribers
Continue reading
Published in 2005:
Most people believe that they should avoid changing their answer when taking multiple choice tests. Virtually all research on this topic, however, has suggested that this strategy is ill-founded: Most answer changes are from incorrect to correct, and people who change their answers usually improve their test scores. Why? …. Changing an answer when one should have stuck with one’s original answer leads to more "if only …" self-recriminations …[making such events] more memorable.
Thinking Helps
If there's an accepted bias toward not changing your answer it stands to reason that most changed answers will be from incorrect to correct. Only a clearly wrong original choice would be changed. If you adopt a strategy of changing your answers, you might find yourself changing from correct to incorrect more often.
If I may deconstruct the argument:
1) Assume the relationship between the T (threshold-for-answer-changing) and C (chance-that-a-change-improves-test-score) isn't sharply discontinuous;2) We know that at current levels of T, C is > 50%therefore:3) On average, people should decrease their levels of T.
Taken literally this reasoning is fallacious. What we want to know is the marginal gain from decreasing our threshold, not whether the average C given the new threshold is >50%.
It would be fair to say that this gives us some evidence that it might not be harmful to further decrease our answer-changing threshold, T.
Of course, it is possible that some of the cited studies have attempted to plot the relationship between T and C (for example, by instructing different students to use different heuristics for when to change one's answer, and plotting the relationship). In which case the quoted passage is fallacious at face value but only because it oversimplifies the point.