I wrote the following (with Aleks Jakulin) to introduce a special issue on Bayesian statistics of the journal Statistica Sinica (volume 17, 422-426). I think the article might be of interest even to dabblers in Bayes, as I try to make explicit some of the political or quasi-political attitudes floating around the world of statistical methodology.
Paulie Fortunato Carlo Schmid-Sutter List of Taekwondo Techniques Pereyaslavets The Tragedy of Macbeth Konoha-tengu Francis Kynaston D'Arcy, Saskatchewan History of Dalsland La Casa de los millones
I would focus as much on subjectivity in the choice of likelihood as in the choice of prior. More generally, with hierarchical models a single prior (or population) distribution can apply to many settings, which somewhat allays the concerns in the second-to-last-paragraph of your comment. See Section 2.8 of Bayesian Data Analysis for an example.
The key insight in Bayesian statistics is that once you have enough data the prior distribution doesn't really matter. The data overwhelm it and you end up with pretty much the same posterior distribution no matter what prior you started with. This insight keeps getting lost. Practictioners naturally hope that the quantity of data that they have managed to collect is sufficient and look to choosing the "correct" prior to get the maximum of information from the data. Rather than worry about getting the "correct" prior, ones method could focus on trying out all the different priors that seem reasonable. If different priors lead to different results one concludes that the data gathered so far are not yet sufficient to decide the issue, and if shortage of time forces a choice one is aware that one is exercising ones judgement, for better or worse, in prejudging the issue by basing ones forced choice on one prior in preference to another.
I propose the use of, to coin a phrase, dialectical priors. In a controversy the two sides may compute different posterior distributions from the same data because they start from different priors. This is a feature not a bug and we should celebrate it as a postive advantage of the Bayesian approach. What has happened is that evidence that seems quite adequate to those whose judgement inclines them to accept a proposition nevertheless seems feeble and unpersuasive to those who were sternly skeptical from the outset. In is natural that doubters require more evidence to persuade them than enthusiasts do. It is unreasonable to expect a methodology to resolve such disputes and it is to the credit of Bayesianism that it sharpens such disputes rather than disguising them.
Things get really interesting if the disputants accept each others priors. This what I think of as "dialectical priors". Then each has a target to shoot for in terms of the quantity and quality of data required to persuade their opponents.
This could go two ways. First, one side could gather sufficient evidence to persuade even using the others side's priors. There is then no room to quibble over the choice of priors. Alternatively, hypothetical calculations may show that a prior distribution is so skeptical that even obviously sufficient evidence cannot overwhelm it. Maintaining such a prior would thus be exposed as an attempt to reject empiricism. I think that the interest lies in the cases where one thinks that a hypothesis is absurd and wish to write in a big fat zero, damning the hypothesis beyond empirical rescue. Well zero will not do, but what small number do you chose a priori? Too small and you appear unreasonably stubborn; no one will care whether you are persuaded. Too large and you will be forced to concede even though you hunger for more data.
There is perhaps a third possibility. Research is difficult and expensive. Perhaps, considering the priors advocated by the parties to the dispute, one may find that realistic amounts of data are unlikely to overwhelm either sides' prior. I need to do some calculations to get a feel for this. I suspect it only arises when there is lots of "noise", for example a treament that is claimed to cut a death rate from 55% to 45%, or a theory in sociology that is claimed to be true on average but is prey to many confounding factors that need to be "averaged out". I believe it is realistic to sometimes reach an impasse in which the available funds for research are insufficient to allow the issue to be decided. It is good for a statistical methodology to reflect this by allowing each side to stick with their a priori judgement.
Learning applied mathematics/statistics (including bayesian analysis) is a high priority for me. It seems to me that one can't think meaningfully about rational decision making (and personal outcome maximization) without some substantial familiarity with mathematical/statistical techniques like bayesian analysis, logical regression, etc. Great post -I'll reread it soon, hopefully for 100% clarity on your points, subpoints, and ideas.
Ace_Bunny
Paulie Fortunato Carlo Schmid-Sutter List of Taekwondo Techniques Pereyaslavets The Tragedy of Macbeth Konoha-tengu Francis Kynaston D'Arcy, Saskatchewan History of Dalsland La Casa de los millones
Alan,
I would focus as much on subjectivity in the choice of likelihood as in the choice of prior. More generally, with hierarchical models a single prior (or population) distribution can apply to many settings, which somewhat allays the concerns in the second-to-last-paragraph of your comment. See Section 2.8 of Bayesian Data Analysis for an example.
The key insight in Bayesian statistics is that once you have enough data the prior distribution doesn't really matter. The data overwhelm it and you end up with pretty much the same posterior distribution no matter what prior you started with. This insight keeps getting lost. Practictioners naturally hope that the quantity of data that they have managed to collect is sufficient and look to choosing the "correct" prior to get the maximum of information from the data. Rather than worry about getting the "correct" prior, ones method could focus on trying out all the different priors that seem reasonable. If different priors lead to different results one concludes that the data gathered so far are not yet sufficient to decide the issue, and if shortage of time forces a choice one is aware that one is exercising ones judgement, for better or worse, in prejudging the issue by basing ones forced choice on one prior in preference to another.
I propose the use of, to coin a phrase, dialectical priors. In a controversy the two sides may compute different posterior distributions from the same data because they start from different priors. This is a feature not a bug and we should celebrate it as a postive advantage of the Bayesian approach. What has happened is that evidence that seems quite adequate to those whose judgement inclines them to accept a proposition nevertheless seems feeble and unpersuasive to those who were sternly skeptical from the outset. In is natural that doubters require more evidence to persuade them than enthusiasts do. It is unreasonable to expect a methodology to resolve such disputes and it is to the credit of Bayesianism that it sharpens such disputes rather than disguising them.
Things get really interesting if the disputants accept each others priors. This what I think of as "dialectical priors". Then each has a target to shoot for in terms of the quantity and quality of data required to persuade their opponents.
This could go two ways. First, one side could gather sufficient evidence to persuade even using the others side's priors. There is then no room to quibble over the choice of priors. Alternatively, hypothetical calculations may show that a prior distribution is so skeptical that even obviously sufficient evidence cannot overwhelm it. Maintaining such a prior would thus be exposed as an attempt to reject empiricism. I think that the interest lies in the cases where one thinks that a hypothesis is absurd and wish to write in a big fat zero, damning the hypothesis beyond empirical rescue. Well zero will not do, but what small number do you chose a priori? Too small and you appear unreasonably stubborn; no one will care whether you are persuaded. Too large and you will be forced to concede even though you hunger for more data.
There is perhaps a third possibility. Research is difficult and expensive. Perhaps, considering the priors advocated by the parties to the dispute, one may find that realistic amounts of data are unlikely to overwhelm either sides' prior. I need to do some calculations to get a feel for this. I suspect it only arises when there is lots of "noise", for example a treament that is claimed to cut a death rate from 55% to 45%, or a theory in sociology that is claimed to be true on average but is prey to many confounding factors that need to be "averaged out". I believe it is realistic to sometimes reach an impasse in which the available funds for research are insufficient to allow the issue to be decided. It is good for a statistical methodology to reflect this by allowing each side to stick with their a priori judgement.
Learning applied mathematics/statistics (including bayesian analysis) is a high priority for me. It seems to me that one can't think meaningfully about rational decision making (and personal outcome maximization) without some substantial familiarity with mathematical/statistical techniques like bayesian analysis, logical regression, etc. Great post -I'll reread it soon, hopefully for 100% clarity on your points, subpoints, and ideas.