Comments on: Chinks In The Bayesian Armor http://www.overcomingbias.com/2007/10/chinks-in-the-b.html Overcoming Bias is economist Robin Hanson’s blog, on honesty, signaling, disagreement, forecasting, and the far future. Sat, 20 Mar 2010 12:32:05 -0400 http://wordpress.org/?v=2.8.5 hourly 1 By: Tim Tyler | http://timtyler.org/ http://www.overcomingbias.com/2007/10/chinks-in-the-b.html#comment-414428 Tim Tyler | http://timtyler.org/ Mon, 07 Jul 2008 21:07:03 +0000 http://prod.ob.trike.com.au/2007/10/chinks-in-the-bayesian-armor.html#comment-414428 The "Chinks In The Bayesian Armor" I would list are: <ul> <li>Inability to deal with undecidable propositions; <li>The problem of the priors; <li>Hume's problem of induction; </ul> Some more possible problems: <a HREF="http://plato.stanford.edu/entries/epistemology-bayesian/#PotPro" rel="nofollow">http://plato.stanford.edu/entries/epistemology-bayesian/#PotPro</A> The “Chinks In The Bayesian Armor” I would list are:

  • Inability to deal with undecidable propositions;
  • The problem of the priors;
  • Hume’s problem of induction;

Some more possible problems:

http://plato.stanford.edu/entries/epistemology-bayesian/#PotPro

]]> By: Dynamically Linked http://www.overcomingbias.com/2007/10/chinks-in-the-b.html#comment-414427 Dynamically Linked Mon, 15 Oct 2007 07:58:12 +0000 http://prod.ob.trike.com.au/2007/10/chinks-in-the-bayesian-armor.html#comment-414427 Robin, I don't think it's more rational to say "If I matter, then X" rather than just "X." Here's my argument. Suppose you hold the following beliefs: - If I matter, then X. - If I don't matter, then not-X. But if you don't matter, then your beliefs don't matter, so you might as well believe "If I don't matter then X." instead. Then you can simplify both of these beliefs into just "X." Robin, I don’t think it’s more rational to say “If I matter, then X” rather than just “X.” Here’s my argument. Suppose you hold the following beliefs:

- If I matter, then X.
- If I don’t matter, then not-X.

But if you don’t matter, then your beliefs don’t matter, so you might as well believe “If I don’t matter then X.” instead. Then you can simplify both of these beliefs into just “X.”

]]> By: Robin Hanson http://www.overcomingbias.com/2007/10/chinks-in-the-b.html#comment-414426 Robin Hanson Sat, 13 Oct 2007 13:29:33 +0000 http://prod.ob.trike.com.au/2007/10/chinks-in-the-bayesian-armor.html#comment-414426 Dynam, it would seem more rational to say "If I matter, i.e., if I exist, am real, and am conscious, then X" rather than just "X." Dynam, it would seem more rational to say “If I matter, i.e., if I exist, am real, and am conscious, then X” rather than just “X.”

]]> By: Dynamically Linked http://www.overcomingbias.com/2007/10/chinks-in-the-b.html#comment-414425 Dynamically Linked Sat, 13 Oct 2007 13:22:32 +0000 http://prod.ob.trike.com.au/2007/10/chinks-in-the-bayesian-armor.html#comment-414425 I think there is a more general framework, within which some of these "exceptions" can be justified. First consider how Bayesianism works in the many-worlds interpretation of QM. If you believe in MWI and also uses Bayesianism to justify your belief in "1+1=2", then you are implicitly condemning your counterparts in many other branches to false beliefs in "1+1=1", "1+1=3", etc., because their mental computers happened to make a mistake while computing 1+1. But you accept this because the measure of those branches are low and you assign less value to the epistemic status of low-measure observers. Similarly, the exceptions "Consciousness", "The Real World", and "Real Stuff" can be justified by assigning less (or no) value to the epistemic status of certain categories of observers. In other words, unconscious and unreal people end up with the false beliefs that they are conscious and real, but this doesn't matter because we don't care that those people end up with false beliefs, just as we don't care that low-measure observers in MWI end up with false beliefs. Of course, I have no idea *why* we don't care about those who are unconscious, unreal, or have low-measure. But I think our beliefs are still justified because who we care about is a subjective value judgment not open to challenge and not requiring further justifications. I think there is a more general framework, within which some of these “exceptions” can be justified. First consider how Bayesianism works in the many-worlds interpretation of QM. If you believe in MWI and also uses Bayesianism to justify your belief in “1+1=2″, then you are implicitly condemning your counterparts in many other branches to false beliefs in “1+1=1″, “1+1=3″, etc., because their mental computers happened to make a mistake while computing 1+1. But you accept this because the measure of those branches are low and you assign less value to the epistemic status of low-measure observers. Similarly, the exceptions “Consciousness”, “The Real World”, and “Real Stuff” can be justified by assigning less (or no) value to the epistemic status of certain categories of observers. In other words, unconscious and unreal people end up with the false beliefs that they are conscious and real, but this doesn’t matter because we don’t care that those people end up with false beliefs, just as we don’t care that low-measure observers in MWI end up with false beliefs.

Of course, I have no idea *why* we don’t care about those who are unconscious, unreal, or have low-measure. But I think our beliefs are still justified because who we care about is a subjective value judgment not open to challenge and not requiring further justifications.

]]> By: Neel Krishnaswami http://www.overcomingbias.com/2007/10/chinks-in-the-b.html#comment-414424 Neel Krishnaswami Fri, 12 Oct 2007 20:42:34 +0000 http://prod.ob.trike.com.au/2007/10/chinks-in-the-bayesian-armor.html#comment-414424 We know that a) mathematicians disagree about what logical implications are acceptable, and b) there is no way to resolve that disagreement. For example, constructive mathematicians reject the principle of the excluded middle -- that is, we reject that P or not-P holds for all P. Furthermore, we cannot construct any experiments to decide whether constructive or classical mathematics is correct, because classical and constructive mathematics agree on all finite examples. (In fact, you can understand classical and constructive mathematics as two <em>different</em> (scientific) inductive generalizations from finite sets.) This poses some deep objections to Bayesianism as a unique standard of rationality. As soon as you want to quantify over spaces of models, you end up needing to form a conception of infinity, of which there isn't a unique best choice. As a result, you can have two different agents who never agree, but also can never present the other with a Dutch Book argument. I think the strongest you can say is that non-Dutch-Book-ability is the test of rationality, of which traditional Bayesianism is one example. We know that a) mathematicians disagree about what logical implications are acceptable, and b) there is no way to resolve that disagreement. For example, constructive mathematicians reject the principle of the excluded middle — that is, we reject that P or not-P holds for all P. Furthermore, we cannot construct any experiments to decide whether constructive or classical mathematics is correct, because classical and constructive mathematics agree on all finite examples. (In fact, you can understand classical and constructive mathematics as two different (scientific) inductive generalizations from finite sets.)

This poses some deep objections to Bayesianism as a unique standard of rationality. As soon as you want to quantify over spaces of models, you end up needing to form a conception of infinity, of which there isn’t a unique best choice. As a result, you can have two different agents who never agree, but also can never present the other with a Dutch Book argument. I think the strongest you can say is that non-Dutch-Book-ability is the test of rationality, of which traditional Bayesianism is one example.

]]> By: Robin Hanson http://www.overcomingbias.com/2007/10/chinks-in-the-b.html#comment-414423 Robin Hanson Thu, 11 Oct 2007 12:41:58 +0000 http://prod.ob.trike.com.au/2007/10/chinks-in-the-bayesian-armor.html#comment-414423 Andrew, I respond <a href="http://www.stat.columbia.edu/~cook/movabletype/archives/2007/10/how_bayesian_am.html#comments" rel="nofollow">at your blog post</a>. Logic, the usual info/prob concept of "know" allows machines to know, even if they have no experiences. Self, I don't see why beliefs from testimony or about other minds would be exceptions. Scott, just yesterday I said intuitions are valid evidence, but I don't see how this resolves the above questions. Sebastian, I lean in your direction, but this post declares me open to hearing more from the other side. Jadagul points in their direction. Andrew, I respond at your blog post.

Logic, the usual info/prob concept of “know” allows machines to know, even if they have no experiences.

Self, I don’t see why beliefs from testimony or about other minds would be exceptions.

Scott, just yesterday I said intuitions are valid evidence, but I don’t see how this resolves the above questions.

Sebastian, I lean in your direction, but this post declares me open to hearing more from the other side. Jadagul points in their direction.

]]> By: Andrew Gelman http://www.overcomingbias.com/2007/10/chinks-in-the-b.html#comment-414422 Andrew Gelman Wed, 10 Oct 2007 22:22:27 +0000 http://prod.ob.trike.com.au/2007/10/chinks-in-the-bayesian-armor.html#comment-414422 Eliezer, Yes, I could agree that the problem is not with Bayesianism but with the models that are being considered. Rather than comparing model A to model B, I'd rather build a third model that includes the two as special cases: that is, continuous model expansion rather than discrete model averaging. Thus, in the ever popular "Are Newton's Laws true?" example, the point is to get away from the binary yes/no response and to recognize that, yes, the laws are false but it can be unclear how to improve them. Bayesian statistics is like other useful theories: it works best when it has good inputs, or when its range of applicability is suitably restricted. Similarly with decision analysis, cost-benefit analysis, or other problem-solving methods. I will say, though, that in practice there will be problems with a statistical model, and that's why we characterize Bayesian data analysis as (1) model building, (2) inference, and (3) model checking. If the model were correct, it would just be step 2, but it's not, so it's not. More discussion <a href="http://www.stat.columbia.edu/~cook/movabletype/archives/2007/10/how_bayesian_am.html" rel="nofollow">here</a>. Eliezer,

Yes, I could agree that the problem is not with Bayesianism but with the models that are being considered. Rather than comparing model A to model B, I’d rather build a third model that includes the two as special cases: that is, continuous model expansion rather than discrete model averaging. Thus, in the ever popular “Are Newton’s Laws true?” example, the point is to get away from the binary yes/no response and to recognize that, yes, the laws are false but it can be unclear how to improve them.

Bayesian statistics is like other useful theories: it works best when it has good inputs, or when its range of applicability is suitably restricted. Similarly with decision analysis, cost-benefit analysis, or other problem-solving methods.

I will say, though, that in practice there will be problems with a statistical model, and that’s why we characterize Bayesian data analysis as (1) model building, (2) inference, and (3) model checking. If the model were correct, it would just be step 2, but it’s not, so it’s not.

More discussion here.

]]> By: Svein Ove http://www.overcomingbias.com/2007/10/chinks-in-the-b.html#comment-414421 Svein Ove Wed, 10 Oct 2007 20:42:22 +0000 http://prod.ob.trike.com.au/2007/10/chinks-in-the-bayesian-armor.html#comment-414421 I'll try the "consciousness" one. The answer seems fairly simple to me - we *don't* have any evidence, but if we're wrong, there won't be anyone around to experience being wrong. As such, it's a reasonable assumption to make - along the lines of only searching for your car keys where you might possibly find them. I’ll try the “consciousness” one.
The answer seems fairly simple to me – we *don’t* have any evidence, but if we’re wrong, there won’t be anyone around to experience being wrong. As such, it’s a reasonable assumption to make – along the lines of only searching for your car keys where you might possibly find them.

]]> By: Jadagul http://www.overcomingbias.com/2007/10/chinks-in-the-b.html#comment-414420 Jadagul Wed, 10 Oct 2007 20:24:58 +0000 http://prod.ob.trike.com.au/2007/10/chinks-in-the-bayesian-armor.html#comment-414420 Sebastian: check out the wikipedia article <a href="http://en.wikipedia.org/wiki/Philosophy_of_mathematics#Contemporary_schools_of_thought" rel="nofollow">here.</a> Roughly speaking, many mathematicians believe that mathematical objects have independent existence and can be perceived directly, or some variation (there are a lot of schools of thought, again read the article). Thus you can engage them in debates about whether the Axiom of Choice, say, is 'true' or 'false'; Banach-Tarski was originally proven in an attempt to say, "See, look at how stupid the results of this axiom are! You don't really believe that." I, on the other hand, incline towards formalism and so don't think axioms have any truth content at all. Sebastian: check out the wikipedia article here. Roughly speaking, many mathematicians believe that mathematical objects have independent existence and can be perceived directly, or some variation (there are a lot of schools of thought, again read the article). Thus you can engage them in debates about whether the Axiom of Choice, say, is ‘true’ or ‘false’; Banach-Tarski was originally proven in an attempt to say, “See, look at how stupid the results of this axiom are! You don’t really believe that.” I, on the other hand, incline towards formalism and so don’t think axioms have any truth content at all.

]]> By: Sebastian Hagen http://www.overcomingbias.com/2007/10/chinks-in-the-b.html#comment-414419 Sebastian Hagen Wed, 10 Oct 2007 16:46:55 +0000 http://prod.ob.trike.com.au/2007/10/chinks-in-the-bayesian-armor.html#comment-414419 A few comments on three of the cases mentioned by Robin: <b>Math and Concept Axioms</b> <i>Some people think ... we know which math axioms are true.</i> What does it mean for a math axiom to be <i>true</i>? I understand what it means for a formal system to be consistent, and that certain formal systems can be more or less useful in modeling certain aspects of our universe, but I don't understand what it means to call a formal system, or a single axiom, <i>true</i>. <b>Basic Moral Claims</b>: Basic moral claims are claims about the optimizing process making the claim, or more concretely about aspects of the physical substrate implementing that process. If I say "X is good" this means that the entity Sebastian Hagen prefers future states of the universe with lots of X over those with less, all else being equal. Since I haven't read up on cognitive science I don't understand the details, but this is ultimately a statement about the structure of my brain. Basic moral statements only make sense when considered in the context of an optimization process making them. <b>Consciousness</b>: I've read up on the concept of <a href="http://en.wikipedia.org/wiki/Philosophical_zombie" rel="nofollow">philosophical zombies</a>, but I still don't really understand the case of physically identical bodies. Afaict this concept deliberately refrains from making any testable predictions. Is there any reason to conclude it is not content-free? A few comments on three of the cases mentioned by Robin:

Math and Concept Axioms
Some people think … we know which math axioms are true.
What does it mean for a math axiom to be true? I understand what it means for a formal system to be consistent, and that certain formal systems can be more or less useful in modeling certain aspects of our universe, but I don’t understand what it means to call a formal system, or a single axiom, true.

Basic Moral Claims:
Basic moral claims are claims about the optimizing process making the claim, or more concretely about aspects of the physical substrate implementing that process.
If I say “X is good” this means that the entity Sebastian Hagen prefers future states of the universe with lots of X over those with less, all else being equal. Since I haven’t read up on cognitive science I don’t understand the details, but this is ultimately a statement about the structure of my brain. Basic moral statements only make sense when considered in the context of an optimization process making them.

Consciousness:
I’ve read up on the concept of philosophical zombies, but I still don’t really understand the case of physically identical bodies. Afaict this concept deliberately refrains from making any testable predictions. Is there any reason to conclude it is not content-free?

]]>