All Hail Info Theory

Hard questions are often hard because different ways to think about them conflict.  When each way seems to have strong support, we are reluctant to choose.  But if we cannot avoid the conflict, choose we must.  For example, last October I wrote:

Our standard ("Bayesian") formal theories of information and probability … are by far the main formal approaches to such issues in physics, economics, computer science, statistics, and philosophy.  … There are, however, a number of claimed exceptions, cases where many people think certain beliefs are justified even though they seem contrary to this standard framework. … I am … tempted to reject all claimed exceptions, but that wouldn’t be fair.  So I’m instead raising the issue and offering a quick survey of claimed exceptions. … The following do not seem to be exceptions: Indexicals … Logical Implications … Here are possible exceptions:  Math and Concept Axioms Basic Moral ClaimsConsciousnessThe Real World Real Stuff … [I could have added religious beliefs to this list.]

Actually, in all these cases it seems it is standard info theory (i.e., info is whatever excludes possibilities) alone that seems to conflict with something else – probability theory is irrelevant.  And it seems to me that: Nothing that seems to conflict with standard info theory is remotely as well established as it is.  So when there is a conflict, info theory must just win.  (More are willing to challenge standard "Bayesian" probability theory – e.g., see Andrew Gelman, Scott Aaronson.) 

Of course we are not absolutely sure of standard info theory. Perhaps our strongest doubts arise from quantum info theory seeming different on its surface.  This is one reason it is so important to sort out the foundations of quantum mechanics. 

Yes, we have many specific intuitions, often very strong, supporting particular beliefs that conflict with info theory.  For example, regarding consciousness most feel we know we are more than just a physical system, having also non-physical "experience."  Regarding morality, most feel we know not just what we personally want, but also a lot about what is absolutely "right," and that this is a fundamentally different thing.  Many also feel they have direct "faith-based" knowledge of religious truths. 

Many feel such intuitions so strongly that they say, "We just can’t reject these beliefs, so we’ll just have to work on revising our theories of info and knowledge to account for how it is we come to know these things."  I think they forget how well established is our info theory. 

We must also admit that in end info theory is itself supported by many other particular intuitions.  We cannot escape relying on intuitions.  But a large set of well-integrated independently-well-supported intuitions should beat fewer more-conceptually-local intuitions, however strong.  Too many natural and strong human intuitions have been just plain wrong.  And info theory is far from local – it has become deeply and widely integrated into many of our best accounts in physics, economics, computer science, statistics, and philosophy.   

We should thus provisionally accept the apparent implications of standard info theory, even when they conflict with other very strong intuitions.  Specifically, we should provisionally accept that:

  • Math shows what axioms imply, but only unconditionally truths about non-math.
  • We have no access to moral truth beyond knowing what we want and why.
  • We have no access to our own consciousness, beyond ordinary interactions.
  • Other possible worlds are just as real as ours.
  • Analytic continuations of accepted theories should be presumed to exist.
  • We have no special access to truths about God or religion.

Info Theory Rules!  I suspect Eliezer agrees with most if not all of this. 

Added 8July: On consciousness I have in mind Chalmers saying that he knows he is conscious, even though a Zombie’s brain would have exactly as the same info as his brain has.  On morals I have in mind people saying they know moral truth even though they see no interaction which could correlate beliefs and moral truth. 

GD Star Rating
loading...
Tagged as:
Trackback URL:
  • mitchell porter

    Robin, how on Earth do you derive your six final propositions from “information theory”? Are you using it as a synonym for Occam’s razor?

  • http://hanson.gmu.edu Robin Hanson

    Mitchell, did you try following the first link?

  • Anatoly Vorobey

    Robin,

    Could you explain what you mean by “formal info theory” or “standard info theory”? You seem to mean by it something well-separated from Bayesian probabilistic reasoning, but I can’t understand what it is, and why it is “as well established as it is”.

    If it’s “information is whatever excludes possibilities”, that doesn’t seem like much, and I’m not sure why it’s as extremely well established as you’re saying it is. Applying this definition to math, are you saying that if we learn tomorrow that Riemann’s hypothesis is true, we won’t thereby receive any information?

    Regarding math: I’m having trouble parsing the first claim: does “unconditionally” modify “shows” or “imply”? Could you rephrase this?

  • Anonymous

    The main problem of information theory as a model of rationality is that it does not have a good way of dealing with (possibly bounded) logical inference and computation. The main exceptions to this are derived from our models of non-deterministic or probabilistic computation, e.g. algorithms for solving NP-complete problems can be naturally modeled as Bayesian search. But the main point still stands.

  • http://hanson.gmu.edu Robin Hanson

    Anon, see this post and this paper.

    Anatoly, finding a proof of Reimann’s hypothesis would surely be info. Math can show that 2+2=4 is true conditional on standard axioms, but it is only unconditionally true about real things to which these axioms apply.

  • Anatoly Vorobey

    Robin, if Riemann’s hypothesis logically follows from the standard axioms, then there is no possibility of its claim being incorrect, and there is no “possible world” in which it doesn’t follow from the axioms; therefore producing a proof of Riemann’s hypothesis from the axioms excludes no possibilities, and seems to not be information according to the definition you propose.

    “Math can show that 2+2=4 is true conditional on standard axioms, but it is only unconditionally true about real things to which these axioms apply.”

    I don’t understand. Are you saying that 2 is a “thing” and 4 is a “thing” about which 2+2=4 is true? If not, if what you’re talking about is not the familiar Platonism of this sort, then what do you mean by “real things to which these axioms apply”? Mathematical axioms do not apply to real things. The powerset axiom is not about physical objects. Is the axiom of choice true “in the real world”?

  • http://hanson.gmu.edu Robin Hanson

    Anatoly, on info about impossibilities, see my answer to Anon. Pennies are real things to which the axioms of 2+2=4 apply.

  • Unknown

    Robin, it’s not clear that your arguments for the position that info theory implies these things are valid.

    For example, “Other possible worlds are just as real as ours” can be shown to be false with as high a degree of probability as you like, by means of anthropic reasoning, if we allow “possible world” to have a wide enough meaning. I’ve made this argument before: let’s admit there is a possible world where my computer explodes in the next ten seconds, one where it begins to hover 10 feet in the air, and so on. There are far more “strange” possibilities than normal possibilities, so if all of these are possible worlds, and all of them are real, then by anthropic reasoning, they should all be equally likely to be experienced by me. The fact that none of these things happen is strong evidence that under this definition of possible world, not all possible worlds are real. It is possible that all possible worlds are real according to some more restricted type of possible world.

    I might be able to think of other arguments regarding the other things as well, also without implying any conflict with info theory.

  • Anatoly Vorobey

    Robin, the axioms from which 2+2=4 follows are e.g. Peano axioms or a standard set theory. These axioms require an infinity of entities in their models. Unless you can show that there is an infinite number of pennies in the world, it is incorrect to say that “pennies are real things to which the axioms of 2+2=4 apply”.

    That’s only one reason this claim is wrong, however. The more important reason is that addition is simply not a physical operation that applies to physical objects like pennies. “Adding pennies” is not strictly speaking an operation performed on physical objects, like, say, “moving pennies” is (this confusion is among other things the source of “paradoxes” like “one raindrop + one raindrop = one raindrop”, which are merely uninteresting misunderstandings).

  • mitchell porter

    Robin, yes I followed the link, and I saw arguments but not information-theoretic arguments. For each of the five “possible exceptions”, you simply ask where the alleged knowledge comes from. That question derives its power from the fact that there is nothing in the natural-scientific ontology capable of being the cause of the alleged knowledge. But if proponents of those positions are willing to bite the bullet and say there are extra sorts of things or extra sorts of causal relations as required, then in principle those new causal avenues of knowledge could be analysed as information channels, just like the naturalistic ones. And I don’t think it is information theory which one uses to judge whether those ontological additions are plausible or possible, except perhaps in one respect: the principle that you don’t postulate causes unless you have unexplained effects hanging around. That is Occam’s razor.

  • http://brokensymmetry.typepad.com Michael F. Martin

    Robin,

    This is an incredibly dangerous path to take. Theory and experiment must live together. Moral intuitions are part of the data set with which theory must engage. When theory has been given privilege over experiment, monstrosities have resulted. Kant should be respected for his willingness to acknowledge that his theories suggested infanticide. Theorists with less integrity have inadvertently caused genocides.

    I don’t care what your theory is — Hegelian, Lukacsian, Information, whatever. Once you fail to acknowledge and engage with difficult to quantify or understand mechanisms that are embodied in everyday wisdom, you fail to be relevant to most of the world.

    If information theory is the one true theory, then it will engage with revealed human experience.

  • michael vassar

    I think that we should taboo “real”, “exist”, etc.

  • http://www.philosophyetc.net Richard

    Robin, you suggest that:
    (1) We should accept that the ‘worlds’ posited by info theory are all equally real.
    (2) Info theory should posit impossible worlds.

    Doesn’t that commit you to the absurd conclusion that there are real contradictions?

  • poke

    I agree with the bullet points except:

    We have no access to moral truth beyond knowing what we want and why.

    I don’t think you can make a strong claim that we know what we want and why. As far as knowing what we want goes, it depends on the connectivity of the brain, and as for why, that would seem to depend on knowing an awful lot about psychology and sociology.

    Other possible worlds are just as real as ours.

    You make the error in the linked post. Descriptions don’t have “the same sort of relations to systems and info in that world that we do to systems and info in our world” they have descriptions of relations to systems and descriptions of info in that world. You can’t have relations between non-real things and therefore cannot use relations between non-real things to argue that they are real. You can only describe the same relations between descriptions of the real world as those described things have in the real world.

  • http://hanson.gmu.edu Robin Hanson

    Unknown, I agree that if one could show that what we see would be very unlikely given the assumption that all possible worlds are real, then we might conclude otherwise. But I am far from convinced this has been shown.

    Anatoly, we do not know whether there are an infinite number of pennies, but even if there are not I’d say 2+2=4 is still true about them.

    Mitchell, yes if one willing to postulate extra things and causal relations, then the conflict goes away.

    Michael M., it is not a matter of data vs. theory, but of less data vs. more data summarized into theory. You seem reluctant to ever reject any strong intuition, no matter what else it might conflict with.

    Richard, the troubling argument by Lewis is that there is nothing internal to a consistent possible world that makes their info about their being real worse that your info about your being real. But for inconsistent worlds, we very definitely can find things internal to their world that shows why their world is not real.

    Poke, I didn’t mean to claim we know a lot about what we want and why.

  • http://brokensymmetry.typepad.com Michael F. Martin

    @Robin

    I am indeed so reluctant. If an intuition is strong, then it must be for some reason. If you don’t know the reason, then you cannot rule out the possibility that your theory is catastrophically wrong.

    The strength of a theory depends crucially on its ability to withstand the assault of contrary intuitions.

  • http://shagbark.livejournal.com Phil Goetz

    I read this post, and the referenced earlier post, and I wasn’t able to figure out in any case what Robin means.

    Can you give us a specific example of a field that claims to contradict information theory, and spell out the contradiction? Ideally, show a calculation for which info theory and the “standard practitioner of the field” get different results.

  • TGGP

    Michael Martin, you cannot rule out the possibility that your intuition(s) is/are catastrophically wrong. How do you decide between them and theory? I think rather than waiting to know what caused our intuitions to be strong (though perhaps wrong) we should not depend on them except insofar as we have reason to believe they are correct.

  • http://www.hopeanon.typepad.com Hopefully Anonymous

    “Yes, we have many specific intuitions, often very strong, supporting particular beliefs that conflict with info theory. For example, regarding consciousness most feel we know we are more than just a physical system, having also non-physical “experience.” ”

    This seems to me to be a strawman characterization of consciousness conundrums. You don’t need me to post or repost the more challenging aspects here, but it would be helpful for you to acknowledge them. I think the bottom line is there is still a big black box, and that waving it away to a debate about physical vs. non-physical systems (since whatever’s in that black box will probably be redefined as part of the physical system, if it doesn’t already lie in what our common 2008 understanding of what the physical system is) reduces it to a strawman that you can pit against “information theory” and equate with “God”, “religion”, and “moral truth” (and I suppose leprecauns and the Easter bunny).

    I’d put understanding consciousness, and the question of whether the answers lie outside of the information theory framework more in an analytical category of “why do we not seem to have encountered other intelligent life” and “what came before the big bang” — in particular, I’d put the apparent individualness, conservation, and the theatre of conscious experience as being in large parts still in a black box.

    This post may be a bit messy -sorry, don’t have time to edit or organize it better.

  • Anatoly Vorobey

    Robin,

    “Anatoly, we do not know whether there are an infinite number of pennies, but even if there are not I’d say 2+2=4 is still true about them.”

    It’s not enough to just say – an argument is more helpful than an unsupported statement. I provided two explaining why it’s incorrect to say that “the axioms of 2+2=4 (not 2+2=4 itself) apply to pennies”.

    But more importantly (to me personally), I’d really like to understand better what you mean by the incredibly well established information theory, separate from Bayesian probabilistic reasoning. It’s a strong claim, and it’d be very useful to me if elaborated – I spent a lot of time trying to understand what information is, without much success. What is the substance of “info theory” as you refer to it, and could you provide references/links/anything to some sort of strong consensus saying that it is very strongly established? (it’s fine if such discussions use completely different words to refer to the same thing, or anything like that).

  • http://shagbark.livejournal.com Phil Goetz

    Robin wrote:

    We should thus provisionally accept the apparent implications of standard info theory, even when they conflict with other very strong intuitions. Specifically, we should provisionally accept that:

    • Math shows what axioms imply, but only unconditionally truths about non-math.
    • We have no access to moral truth beyond knowing what we want and why.
    • We have no access to our own consciousness, beyond ordinary interactions.
    • Other possible worlds are just as real as ours.
    • Analytic continuations of accepted theories should be presumed to exist.
    • We have no special access to truths about God or religion.

    reliability(info theory) > reliability(intuitions) > reliability(connections Robin is making between intuitions and info theory)

  • Unknown

    Phil, that was exactly the point I was making: the problem is Robin’s argument attempting to show that these things conflict with info theory.

  • http://brokensymmetry.typepad.com Michael F. Martin

    @TGGP

    I think rather than waiting to know what caused our intuitions to be strong (though perhaps wrong) we should not depend on them except insofar as we have reason to believe they are correct.

    I understand the temptation. (It’s the same that Robin has fallen into here.) My point is that we should never choose to ignore an intuition — especially a strong one. If our theory permits us to articulate a reason why an intuition should be ignored or is irrelevant, fine. But then we need to be hyper-sensitive to any data that later emerges in contradiction to our theory, but consistent with the intuition.

    When we selectively ignore intuitions in order to advance theoretical goals, we literally isolate ourselves. Isolation is dangerous — literally.

  • http://hanson.gmu.edu Robin Hanson

    Michael, choosing info theory over a specific intuition when drawing a conclusion is not at all the same as “ignoring” that intuition.

    Hopefully, a one line description pretty much has to be a “strawman.”

  • http://brokensymmetry.typepad.com Michael F. Martin

    Robin,

    I’m not trying to be stubborn here. I really just don’t understand how or why a theory would ever have to choose one intuition over another. Why can’t a theory explain what it can explain, and leave the rest as unresolved mysteries for future research? Maybe we’re not disagreeing here and I simply misunderstood what you were arguing for. But I am stubborn on the point that we should all get used to not being able to understand and explain everything around us, no matter how complete our theories.

    “Choosing” among intuitions is a very slippery slope that we don’t want to step onto. We’re better off getting pulled back and forth around an unstable equilibrium at the top of the hill sometimes.

  • http://profile.typekey.com/halfinney/ Hal Finney

    How about prior probabilities in Bayesian reasoning and information theory? They play a role analogous to the axioms in a mathematical system. They are inputs into the information theory process and seem ultimately to rely on intuition. It is quite mysterious to me where priors come from, and seems unfortunate to have such an instability at the foundation of the Bayesian reasoning system.

  • http://hanson.gmu.edu Robin Hanson

    Seed my added above.

    Hal, info theory alone just specifies the set of states consistent with one’s info, but provides no measure on that set. It is probability theory that goes further to use a measure.

  • http://brokensymmetry.typepad.com Michael F. Martin

    @Hal

    Maybe we should take the existence of priors as a hint at our need to engage — nay embrace — strong intuitions. What do we really know if we don’t know our priors? But really the priors should not be a mystery, they’re an inherited view of the world — inarticulate, perhaps off the mark, but nonetheless reflecting the conserved wisdom of generations and generations of people before us.

  • http://brokensymmetry.typepad.com Michael F. Martin

    @Robin

    Well that resolves the circularity — but at the expense of being able to articulate why info theory should be favored. Which brings us back, I believe, to my original question. Well this was fun. 🙂

  • Cyan

    It is quite mysterious to me where priors come from, and seems unfortunate to have such an instability at the foundation of the Bayesian reasoning system.

    It really is unfortunate — and even more unfortunate is the fact that all reasoning systems have that instability. Even the PAC framework and SVMs are grounded in assumptions about the data-generating mechanism. I’m not (100 – ɛ)% sure of this, but my understanding is that the NFL theorems imply that learning or optimization is pretty much impossible without making some kind of structural assumption.

    I personally don’t worry too much about where priors come from in the general sense; as Andrew Gelman says, they come from the same place likelihoods come from. (I do carefully consider the appropriateness of the priors and likelihoods I actually put to use.)

  • http://www.weidai.com Wei Dai

    Hal, I have a couple of ideas where priors come from, and I’d be interested to hear your thoughts on them.

    Robin, I really like this observation that many interesting implications follow just from the idea that information is whatever allows us to exclude possibilities, and do not depend on the more controversial parts of Bayesianism. Thanks!