Let me try an experiment: using a blog post to develop a taxonomy. Here I'll try to develop a list/taxonomy of (at least semi-coherent) answers to the question I posed yesterday: why is it harder to formally predict pasts, versus futures (from presents)? Mostly these are explanations of the "past hypothesis", but I'm trying to stay open-minded toward a wide range of explanations.
Robin, looking for counter-examples is a useful technique for understanding and judging claims that are not backed by formal arguments, which I admitted are not available. You don't have to prove to my satisfaction any counter-examples you might find. Feel free to state them informally, or just use them to privately update your own beliefs. And as far as I can tell, the claims under discussion are already stated in simple language that doesn't require any specific focus to understand.
Wei, your area is not my focus, so I'm not going to take the time to prove a counter-example, or even to figure out what you mean precisely enough to know what to you would count as a count-example. It is up to the proponents of such claims to offer arguments in their favor.
Robin, do you have a counterexample of a low-entropy state that a physicist has considered, but can't be sampled by a short Monte Carlo program, or a mapping between bit strings and physical states that you consider reasonable, but can't be transformed into the one I gave by a short program?
Wei, if your claim is that there is a universal prior over bit strings, and a mapping from bit strings to physical states such that some low entropy states have high probability, I can accept it. But if your claim is more like that all priors and all mappings make all low entropy states have high probability, you need to do much more than outline an example of one prior, one mapping, and one state.
Robin, this one example is sufficient to explain the "past hypothesis". If you give even a small weight to this particular prior in your actual prior, then your posterior belief, conditioned on your current observations, will be that with high probability you are in a universe at a time coordinate with lower entropy in one direction, and higher entropy in the other.
But my argument applies to any mapping that can be transformed into the particular one I proposed by a short computer program, which should cover all "reasonable" mappings. It also applies to any low-entropy state that can be sampled by short Monte Carlo programs, which should cover most low-entropy states that physicists might consider.
Wei, if your claim is that there is a universal prior over bit strings, and a mapping from bit strings to physical states such that some low entropy states have high probability, I can accept it. But if your claim is more like that all priors and all mappings make all low entropy states have high probability, you need to do much more than outline an example of one prior, one mapping, and one state.
Wei, Bennett says that to specify a state, you can specify a macrostate, and then specify the exact state within it.
Yes.
For lower entropy macrostates, it may take less to specify the exact state within, but it takes more to specify the macrostate itself. Relative to equilibrium distribution expectations, this doesn't make it easier to describe such states.
You're not supposed to specify a macrostate relative to the equilibrium distribution, but rather specify it as a computer program, relative to a universal Turing machine. One page 938 in Bennett's paper, there's a paragraph that goes "We needs to say in more detail what it means to describe a distribution [i.e. macrostate] ... a Monte Carlo program for sampling some distribution q not too different from p". Consider such a program for sampling the low-entropy initial macrostate proposed by the inflation hypothesis. It may be somewhat longer than the program for sampling the equilibrium distribution, but surely that's more than made-up by the vastly smaller number of bits needed to specify a microstate within the low-entropy macrostate.
Yes, if you have non-equilibrium expectations that make low entropy macrostates more likely, you can take advantage of this to create shorter descriptions of exact states within such macrostates. But the whole question here was explaining why such expectations make sense; you can't assume them and then think you've proved why they make sense.
The idea is, some states are more likely than others because they have shorter descriptions relative to a universal Turing machine. You're right that this deviates from equilibrium expectations, but having this kind of prior seems to work, in the sense of giving sensible predictions, whereas equilibrium expectations give nonsensical predictions, as you've observed. If you need further motivations, there's the appeal to Occam's Razor and Schmidhuber's suggestion that reality is directly structured to favor universes described by short programs. I consider the question of "why does the universal prior make sense?" to be still open, but it clearly makes more sense than equilibrium expectations.
I gotta guess that quantum mechanics is involved in the solution here Robin. Complex things are built from simple things - a general principle? - if so the universe at the beginning was in the simplest possible state. What is the simplest state of a QM wave function? Maximal coherence surely? So decoherence is associated with increasingly complex quantum branching...think an ever more complex branching tree in many-worlds, needs more info to specify. Decoherence represents info loss in a particular observer branch, and thus increasing entropy. And, it is more complex to specify.
Any way, its important to get this solved. I'm damn sure there's a whopping rebuttal of all of Yudkowsky's ideas at the end of this, and if I find out I'm right about my radical postulates (Bayes just a special case of something more general like analogy formation, beauty a universal terminal value etc etc) I'll be trumpeting for all eternity by golly.
Wei, Bennett says that to specify a state, you can specify a macrostate, and then specify the exact state within it. For lower entropy macrostates, it may take less to specify the exact state within, but it takes more to specify the macrostate itself. Relative to equilibrium distribution expectations, this doesn't make it easier to describe such states. Yes, if you have non-equilibrium expectations that make low entropy macrostates more likely, you can take advantage of this to create shorter descriptions of exact states within such macrostates. But the whole question here was explaining why such expectations make sense; you can't assume them and then think you've proved why they make sense.
Robin, you're right, that section in Cover and Thomas doesn't quite show what I need. Instead, try section 6, "Algorithmic Entropy and Thermodynamics", of C.H. Bennett's The Thermodynamics of Computation - A Review, or Chapter 8, "Physics, Information, and Computation", of Li and Vitanyi's An Introduction to Kolmogorov Complexity and Its Applications.
The hope is that string cosmology will somehow determine the initial conditions.
Um, not exactly. There are also competing theories that look 'the same' for sufficiently diverse definitions of 'same'. One possibility is that time is not a first-order presupposition, but rather an emergent phenomena. Another one is that travel backwards in time is impossible.
Robin, the relationship between entropy and description length is a basic result of Algorithmic Information Theory. See section 7.3 of Cover and Thomas if you still have it handy (or view it at http://www.amazon.com/Eleme.... Informally, a lower-entropy state exhibits more regularities (think of ice crystals vs. liquid water), which can be compressed into a shorter description.
Robin currently lists 9 explanations. Three rely on a "local time asymmetry" (here I include the one about state mappings). Two say we live after "a big local ebb" in matter entropy. Two talk about space-time boundary conditions. One appeals to magic (God), one to semantics (different questions).
So, reorganizing on the basis of type of explanation, I'd propose classifying as follows:
1. This Is All Just One Big Fluctuation, High Entropy Is The Norm2. The Big Bang Was Low-Entropy For Some Special Reason3. There Is A Local Dynamical Asymmetry Which Decreases Entropy If You Work Backwards4. Other
(1) is Boltzmann's original idea, but people think the universe is unnecessarily large to be a fluctuation - why not have only one galaxy, or only one solar system? (2) has some defenders, but no-one is able to say *why* it was low entropy, except to say it's a basic law or an act of God. (3) sounds like it could explain the difference, except it doesn't explain why the past started *that* low in entropy. Why couldn't, why shouldn't, a universe with this posited time-asymmetric dynamics still start in a high-entropy state?
Inflation comes up in these discussions, but I read that inflation itself requires unusual initial conditions.
Under the original post, Robin asked how someone could hope that string theory provides the answer. The hope is that string cosmology will somehow determine the initial conditions. However, at this time I only see people translating ideas from pre-string quantum cosmology into the string context, but no specifically stringy considerations that can decide between them. I suppose that if a particular choice of cosmic initial conditions happened to favor a long-term string ground state that resembles the physics we observe, that would be regarded as a strong post-hoc reason for believing that this is the right choice. But we would still be lacking an explanation for it.
Robin, if a universe has a low-entropy state at some time coordinate (with that time coordinate also having a short description), you can give a short description of this universe by describing the low-entropy state, the time coordinate at which it occurs, and the function for mapping states across time.
Robin, looking for counter-examples is a useful technique for understanding and judging claims that are not backed by formal arguments, which I admitted are not available. You don't have to prove to my satisfaction any counter-examples you might find. Feel free to state them informally, or just use them to privately update your own beliefs. And as far as I can tell, the claims under discussion are already stated in simple language that doesn't require any specific focus to understand.
Wei, your area is not my focus, so I'm not going to take the time to prove a counter-example, or even to figure out what you mean precisely enough to know what to you would count as a count-example. It is up to the proponents of such claims to offer arguments in their favor.
Robin, do you have a counterexample of a low-entropy state that a physicist has considered, but can't be sampled by a short Monte Carlo program, or a mapping between bit strings and physical states that you consider reasonable, but can't be transformed into the one I gave by a short program?
Wei, we already believe the past was low entropy. I won't just take your word regarding your "reasonable" and "most" claims.
Wei, if your claim is that there is a universal prior over bit strings, and a mapping from bit strings to physical states such that some low entropy states have high probability, I can accept it. But if your claim is more like that all priors and all mappings make all low entropy states have high probability, you need to do much more than outline an example of one prior, one mapping, and one state.
Robin, this one example is sufficient to explain the "past hypothesis". If you give even a small weight to this particular prior in your actual prior, then your posterior belief, conditioned on your current observations, will be that with high probability you are in a universe at a time coordinate with lower entropy in one direction, and higher entropy in the other.
But my argument applies to any mapping that can be transformed into the particular one I proposed by a short computer program, which should cover all "reasonable" mappings. It also applies to any low-entropy state that can be sampled by short Monte Carlo programs, which should cover most low-entropy states that physicists might consider.
Wei, if your claim is that there is a universal prior over bit strings, and a mapping from bit strings to physical states such that some low entropy states have high probability, I can accept it. But if your claim is more like that all priors and all mappings make all low entropy states have high probability, you need to do much more than outline an example of one prior, one mapping, and one state.
Wei, Bennett says that to specify a state, you can specify a macrostate, and then specify the exact state within it.
Yes.
For lower entropy macrostates, it may take less to specify the exact state within, but it takes more to specify the macrostate itself. Relative to equilibrium distribution expectations, this doesn't make it easier to describe such states.
You're not supposed to specify a macrostate relative to the equilibrium distribution, but rather specify it as a computer program, relative to a universal Turing machine. One page 938 in Bennett's paper, there's a paragraph that goes "We needs to say in more detail what it means to describe a distribution [i.e. macrostate] ... a Monte Carlo program for sampling some distribution q not too different from p". Consider such a program for sampling the low-entropy initial macrostate proposed by the inflation hypothesis. It may be somewhat longer than the program for sampling the equilibrium distribution, but surely that's more than made-up by the vastly smaller number of bits needed to specify a microstate within the low-entropy macrostate.
Yes, if you have non-equilibrium expectations that make low entropy macrostates more likely, you can take advantage of this to create shorter descriptions of exact states within such macrostates. But the whole question here was explaining why such expectations make sense; you can't assume them and then think you've proved why they make sense.
The idea is, some states are more likely than others because they have shorter descriptions relative to a universal Turing machine. You're right that this deviates from equilibrium expectations, but having this kind of prior seems to work, in the sense of giving sensible predictions, whereas equilibrium expectations give nonsensical predictions, as you've observed. If you need further motivations, there's the appeal to Occam's Razor and Schmidhuber's suggestion that reality is directly structured to favor universes described by short programs. I consider the question of "why does the universal prior make sense?" to be still open, but it clearly makes more sense than equilibrium expectations.
I gotta guess that quantum mechanics is involved in the solution here Robin. Complex things are built from simple things - a general principle? - if so the universe at the beginning was in the simplest possible state. What is the simplest state of a QM wave function? Maximal coherence surely? So decoherence is associated with increasingly complex quantum branching...think an ever more complex branching tree in many-worlds, needs more info to specify. Decoherence represents info loss in a particular observer branch, and thus increasing entropy. And, it is more complex to specify.
Any way, its important to get this solved. I'm damn sure there's a whopping rebuttal of all of Yudkowsky's ideas at the end of this, and if I find out I'm right about my radical postulates (Bayes just a special case of something more general like analogy formation, beauty a universal terminal value etc etc) I'll be trumpeting for all eternity by golly.
Wei, Bennett says that to specify a state, you can specify a macrostate, and then specify the exact state within it. For lower entropy macrostates, it may take less to specify the exact state within, but it takes more to specify the macrostate itself. Relative to equilibrium distribution expectations, this doesn't make it easier to describe such states. Yes, if you have non-equilibrium expectations that make low entropy macrostates more likely, you can take advantage of this to create shorter descriptions of exact states within such macrostates. But the whole question here was explaining why such expectations make sense; you can't assume them and then think you've proved why they make sense.
Robin, you're right, that section in Cover and Thomas doesn't quite show what I need. Instead, try section 6, "Algorithmic Entropy and Thermodynamics", of C.H. Bennett's The Thermodynamics of Computation - A Review, or Chapter 8, "Physics, Information, and Computation", of Li and Vitanyi's An Introduction to Kolmogorov Complexity and Its Applications.
The hope is that string cosmology will somehow determine the initial conditions.
Um, not exactly. There are also competing theories that look 'the same' for sufficiently diverse definitions of 'same'. One possibility is that time is not a first-order presupposition, but rather an emergent phenomena. Another one is that travel backwards in time is impossible.
Wei, that section only discusses bit strings, not physical universes. Some low entropy states may exhibit regularities, but most may not.
Mitchell, your summary is reasonable.
Robin, the relationship between entropy and description length is a basic result of Algorithmic Information Theory. See section 7.3 of Cover and Thomas if you still have it handy (or view it at http://www.amazon.com/Eleme.... Informally, a lower-entropy state exhibits more regularities (think of ice crystals vs. liquid water), which can be compressed into a shorter description.
Robin currently lists 9 explanations. Three rely on a "local time asymmetry" (here I include the one about state mappings). Two say we live after "a big local ebb" in matter entropy. Two talk about space-time boundary conditions. One appeals to magic (God), one to semantics (different questions).
So, reorganizing on the basis of type of explanation, I'd propose classifying as follows:
1. This Is All Just One Big Fluctuation, High Entropy Is The Norm2. The Big Bang Was Low-Entropy For Some Special Reason3. There Is A Local Dynamical Asymmetry Which Decreases Entropy If You Work Backwards4. Other
(1) is Boltzmann's original idea, but people think the universe is unnecessarily large to be a fluctuation - why not have only one galaxy, or only one solar system? (2) has some defenders, but no-one is able to say *why* it was low entropy, except to say it's a basic law or an act of God. (3) sounds like it could explain the difference, except it doesn't explain why the past started *that* low in entropy. Why couldn't, why shouldn't, a universe with this posited time-asymmetric dynamics still start in a high-entropy state?
Inflation comes up in these discussions, but I read that inflation itself requires unusual initial conditions.
Under the original post, Robin asked how someone could hope that string theory provides the answer. The hope is that string cosmology will somehow determine the initial conditions. However, at this time I only see people translating ideas from pre-string quantum cosmology into the string context, but no specifically stringy considerations that can decide between them. I suppose that if a particular choice of cosmic initial conditions happened to favor a long-term string ground state that resembles the physics we observe, that would be regarded as a strong post-hoc reason for believing that this is the right choice. But we would still be lacking an explanation for it.
Wei, why is a description of a low entropy state shorter than for other states?
Robin, if a universe has a low-entropy state at some time coordinate (with that time coordinate also having a short description), you can give a short description of this universe by describing the low-entropy state, the time coordinate at which it occurs, and the function for mapping states across time.