In my debate with Bryan Caplan (vid here), his position was a strong audience favorite before, and less so after. When I heard that Tyler Cowen just had the same experience in his debate, I suggested to the lunch crowd that maybe in general debates move audiences toward a 50/50 opinion split. The org that sponsored Tyler’s debate has done
Perhaps the solution is to simply debate to audiences that are overwhelmingly composed of the opposition. If the before verdict is unanimous, then conversion can only go one way.
Perhaps it's better to view debates as providing social evidence in both directions, rather than anything actually persuasive.
Robin, I second Eliezer's desire to see a list of topics. If counterintuitive positions tend to be behind at first and ahead at the end, it may be a sign that debates are valuable.
About the idea that it's easier to steal votes from the other side if they have a large initial lead: shouldn't the arguments of a position with, for example, 90% support be good enough to convince 9 people for every 1 the other side convinces? If they're not, why does the position deserve to have 90% support?
I was confused by the charts at first, too, but I think I've figured it out.
Each debate reports the polling results before and after, as percent favoring, percent opposed, and percent undecided. The "contrarian" position is whichever has the smaller percentage supporting it initially, and the "favorite" position is the one with the larger percentage. Because the percent undecided changes before and after, contrarian and favorite positions do not move the same amount or even necessarily the same direction. So we see rather different effects in the two charts.
The red lines are not fits, they are the X=Y lines; if people's positions did not change at all, all of the points would be on a red line.
One thing I saw (not shown in the charts) is that undecided seems to (at least, almost) always decrease. People feel that they are becoming more certain in their views. That argues against interpretations where people get confused by being exposed to both sides.
It seems plausible that much of the change in position comes from undecided people adopting one of the two sides. That wouldn't explain all of it but it could explain a lot. Taborek argues that the before-undecided are lying, and were not really undecided. (Maybe they are following a social convention of adopting an unbiased stance before hearing the arguments.) Then you have to assume that these pseudo-undecideds have more moderate views than those who are willing to admit to having made up their minds beforehand.
Visually, the effect looks stronger with the contrarian than the favorite position. That would mean (in this interpretation) that people with the unpopular view are more likely to lie and claim to be undecided beforehand. That could make sense if people are used to hiding unpopular views.
My guess is that the post-debate opinion needs to be measured at a later time.
I would expect a debate between two qualified opponents to present a mutually comparable number of arguments that the presenters have researched and understand, to an audience who does not. As a result, the real state of people's minds after a debate is not a changed opinion, it is confusion. When asked what opinion they now lean to, people will respond as if they were asked the question "which side's arguments do you recall the most right now". If the debaters were equally good, this is probably a linear function of how much time each debater got to speak.
Weighing the merits of substantive arguments is a bit like compiling code. It requires time, attention and precision, and cannot be done on the fly during a debate.
Measure people's opinions a week or a month after the debate, and you will know how much they were really affected.
Robin, are the results of the pre-debate votes know to the audience? Your explanation would predict an equalizing effect even if the audience did not know or were mistaken about which view was initially most popular, while Alex’s preferred explanation depends on the audience knowing which position is the “underdog.” (On Alex’s view, the audience should instead switch toward the favorite if they mislead about which is the favorite initially.)
Also, it looks like the source data includes percent undecided. Does your analysis change if you consider these percents? (If the only effect of the debate were to cause undecided people to commit randomly, then that would increase the percentage for the underdog position, but probably should not be interpreted as an increase in support.)
Acinonys, there were probably a couple of people being neutral both before and after the debate, so the sum doesn't give 100% and favourite can have less than 50%.
Nevertheless, did the audience know whether they are on the favourite or contrarian side? Are there any data about strength of their beliefs before and after the debate? Has the audience consisted of people who are aware of cognitive biases or it has been a typical sample of general public?
@Stephen: Yes, you've reinforced my point. And to say "an arrow in the direction of the more probably correct answer" rather than "...the more likely correct answer" were it intentional, is the distinction I've been trying to impress on our dear Archimedes.
But also consider how cheap it is to dismiss choices as false dichotomies.
When a living thing kills another living thing, in some way it subsumes its useful material and leaves the rest. It's the same with sides in a debate: the victor is not categorically right, even if he gets a consensus victory. If he is smart he will adopt the better points of his opponents argument.
The implication is that anyone who commits to a belief is committed to being wrong. The truth-promoting strategy is "strong opinions, loosely held." The upshot is that preference is only an arrow in the direction of the more probably correct answer and to refine our answers we must gather new data and find new patterns in old data.
Consider how cheap is a binary choice, even with regard to a complex high-dimensional issue. Consider how easy it is to say "you're either with us or against us" when in the bigger picture it's clear we have much more in common than not. Consider the tendency toward bifurcation along political lines, right and wrong, good and evil, god and devil...Why do the branches of a tree bifurcate rather than something more complex?
Consider a simple electronic feedback loop, controlling for only a single variable such as voltage. As it becomes "locked", it will tend toward bipolar symmetrical variation about its set point.
A more complex system, dancing in many dimensions, may be considered not "locked" but adapted, such that its dynamical nature tends to mirror that of its environment. Is it surprising that studies of twins, in regard to traits of adaptation, often show variation correlated roughly 50% nature, 50% environment?
What does this imply for prediction of preferences in general, as we move from the common root of our evolved nature, to the tips of our leaves exploring their adjacent possible?
Shouldn't the two plots be the same data from different view points? But then there wouldn't be just one data point below the neutral line in the first and many in the second.
You speak about 27 debates but there are 26 points in the first and 25 in the second chart, if i did count right.
What do you mean by favourite? I thought " >50% percent for his position before", but most points in the second graph have a x-value < 50
If Alex is correct, debates will not improve public opinion. It will move opinions toward a 50-50 split regardless of the relative merits of the positions.
wait, if audiences move towards 50-50 then more debates isn't necessarily a good thing. You could engineer a series of debates that lead an audience further and further from the truth stepwise.
On a continuum from "Where should we eat dinner?" to "Gordian knot" your topics of debate tend toward the latter. On top of that, there is no real incentive to reach consensus in your forums.
In most "real" decisions, problems are simple and consensus is either necessary or preferred. In those cases I'd bet you see trends favoring the initial majority.
I would also take a look at the sample: is it representative of the public or is it representative of the type of people who read sites like OB? I think the second of those groups is way more likely to be composed of people who are able to abandon beliefs in light of new evidence. Given two beliefs with no known objectively correct answer and approximately the same amount of evidence for each, an initial minority should always expect to gain ground in debates among such people.
The link to Tyler Cowen's debate no longer works, the current url is this.
Perhaps the solution is to simply debate to audiences that are overwhelmingly composed of the opposition. If the before verdict is unanimous, then conversion can only go one way.
Perhaps it's better to view debates as providing social evidence in both directions, rather than anything actually persuasive.
Hal is right about the details, for anyone else who was confused.
Eliezer and Gary, follow the link for the titles.
blink, watch or listen to a debate to find out what folks were told.
Robin, I second Eliezer's desire to see a list of topics. If counterintuitive positions tend to be behind at first and ahead at the end, it may be a sign that debates are valuable.
About the idea that it's easier to steal votes from the other side if they have a large initial lead: shouldn't the arguments of a position with, for example, 90% support be good enough to convince 9 people for every 1 the other side convinces? If they're not, why does the position deserve to have 90% support?
Great video - shouldn't it have had its own thread, though?
...and where is *your* "economist's hat"? Please provide pictures! ;-)
I was confused by the charts at first, too, but I think I've figured it out.
Each debate reports the polling results before and after, as percent favoring, percent opposed, and percent undecided. The "contrarian" position is whichever has the smaller percentage supporting it initially, and the "favorite" position is the one with the larger percentage. Because the percent undecided changes before and after, contrarian and favorite positions do not move the same amount or even necessarily the same direction. So we see rather different effects in the two charts.
The red lines are not fits, they are the X=Y lines; if people's positions did not change at all, all of the points would be on a red line.
One thing I saw (not shown in the charts) is that undecided seems to (at least, almost) always decrease. People feel that they are becoming more certain in their views. That argues against interpretations where people get confused by being exposed to both sides.
It seems plausible that much of the change in position comes from undecided people adopting one of the two sides. That wouldn't explain all of it but it could explain a lot. Taborek argues that the before-undecided are lying, and were not really undecided. (Maybe they are following a social convention of adopting an unbiased stance before hearing the arguments.) Then you have to assume that these pseudo-undecideds have more moderate views than those who are willing to admit to having made up their minds beforehand.
Visually, the effect looks stronger with the contrarian than the favorite position. That would mean (in this interpretation) that people with the unpopular view are more likely to lie and claim to be undecided beforehand. That could make sense if people are used to hiding unpopular views.
My guess is that the post-debate opinion needs to be measured at a later time.
I would expect a debate between two qualified opponents to present a mutually comparable number of arguments that the presenters have researched and understand, to an audience who does not. As a result, the real state of people's minds after a debate is not a changed opinion, it is confusion. When asked what opinion they now lean to, people will respond as if they were asked the question "which side's arguments do you recall the most right now". If the debaters were equally good, this is probably a linear function of how much time each debater got to speak.
Weighing the merits of substantive arguments is a bit like compiling code. It requires time, attention and precision, and cannot be done on the fly during a debate.
Measure people's opinions a week or a month after the debate, and you will know how much they were really affected.
Robin, are the results of the pre-debate votes know to the audience? Your explanation would predict an equalizing effect even if the audience did not know or were mistaken about which view was initially most popular, while Alex’s preferred explanation depends on the audience knowing which position is the “underdog.” (On Alex’s view, the audience should instead switch toward the favorite if they mislead about which is the favorite initially.)
Also, it looks like the source data includes percent undecided. Does your analysis change if you consider these percents? (If the only effect of the debate were to cause undecided people to commit randomly, then that would increase the percentage for the underdog position, but probably should not be interpreted as an increase in support.)
Acinonys, there were probably a couple of people being neutral both before and after the debate, so the sum doesn't give 100% and favourite can have less than 50%.
Nevertheless, did the audience know whether they are on the favourite or contrarian side? Are there any data about strength of their beliefs before and after the debate? Has the audience consisted of people who are aware of cognitive biases or it has been a typical sample of general public?
@Stephen: Yes, you've reinforced my point. And to say "an arrow in the direction of the more probably correct answer" rather than "...the more likely correct answer" were it intentional, is the distinction I've been trying to impress on our dear Archimedes.
But also consider how cheap it is to dismiss choices as false dichotomies.
When a living thing kills another living thing, in some way it subsumes its useful material and leaves the rest. It's the same with sides in a debate: the victor is not categorically right, even if he gets a consensus victory. If he is smart he will adopt the better points of his opponents argument.
The implication is that anyone who commits to a belief is committed to being wrong. The truth-promoting strategy is "strong opinions, loosely held." The upshot is that preference is only an arrow in the direction of the more probably correct answer and to refine our answers we must gather new data and find new patterns in old data.
Consider how cheap is a binary choice, even with regard to a complex high-dimensional issue. Consider how easy it is to say "you're either with us or against us" when in the bigger picture it's clear we have much more in common than not. Consider the tendency toward bifurcation along political lines, right and wrong, good and evil, god and devil...Why do the branches of a tree bifurcate rather than something more complex?
Consider a simple electronic feedback loop, controlling for only a single variable such as voltage. As it becomes "locked", it will tend toward bipolar symmetrical variation about its set point.
A more complex system, dancing in many dimensions, may be considered not "locked" but adapted, such that its dynamical nature tends to mirror that of its environment. Is it surprising that studies of twins, in regard to traits of adaptation, often show variation correlated roughly 50% nature, 50% environment?
What does this imply for prediction of preferences in general, as we move from the common root of our evolved nature, to the tips of our leaves exploring their adjacent possible?
I don't really understand the charts.
Shouldn't the two plots be the same data from different view points? But then there wouldn't be just one data point below the neutral line in the first and many in the second.
You speak about 27 debates but there are 26 points in the first and 25 in the second chart, if i did count right.
What do you mean by favourite? I thought " >50% percent for his position before", but most points in the second graph have a x-value < 50
And why don't the axis have the same scale?
What did I miss?
If Alex is correct, debates will not improve public opinion. It will move opinions toward a 50-50 split regardless of the relative merits of the positions.
wait, if audiences move towards 50-50 then more debates isn't necessarily a good thing. You could engineer a series of debates that lead an audience further and further from the truth stepwise.
On a continuum from "Where should we eat dinner?" to "Gordian knot" your topics of debate tend toward the latter. On top of that, there is no real incentive to reach consensus in your forums.
In most "real" decisions, problems are simple and consensus is either necessary or preferred. In those cases I'd bet you see trends favoring the initial majority.
I would also take a look at the sample: is it representative of the public or is it representative of the type of people who read sites like OB? I think the second of those groups is way more likely to be composed of people who are able to abandon beliefs in light of new evidence. Given two beliefs with no known objectively correct answer and approximately the same amount of evidence for each, an initial minority should always expect to gain ground in debates among such people.