Back in 2008 Eliezer Yudkowsky blogged here with me, and over several months we debated his concept of “AI foom.” In 2011 we debated the subject in person. Yudkowsky’s research institute has now put those blog posts and a transcript of that debate together in a free book:

Well, a process that changes infinitely fast is a bit like an infinite density in time...

"I think what you're saying is that physical continuity (nonquantization of time or space) implies that infinitisimal time spans exist."

Yes, but even if infinitesimal time spans exist that does not have to mean any physical process works in infinitesimal time steps.

"I think assuming continuity implies actual infinitesimal time spans create Zeno's paradox"

Or processes just take an infinite number of infinitesimal steps at a time.

This reminds me of an Isaac Asimov short story where people have discovered how to travel to alternative universes. They reckon there are an infinite number of them so people start claiming alternative Earths for themselves and they expect to never encounter an alternative humanity. Than one day they do because the alternative Earths with humans on them are a fraction of all the worlds: the probability of meeting them is actually finite. Perhaps nature solves Zeno's paradox in a similar way.

you're definitely not going to get infinitely many worlds without allowing for a type of infinity within every world.

Doesn't that contradict your earlier claim:

"It depends on whether you believe states can change over infinitesimal timespans."

I think what you're saying is that physical continuity (nonquantization of time or space) implies that infinitisimal time spans exist. (I think assuming continuity implies actual infinitesimal time spans create Zeno's paradox - http://tinyurl.com/b9kn4tb )

Indeed you don't, infinite energy ranges or infinite densities also do the trick, it's just one way to get to an infinite number of possible worlds, but you're definitely not going to get infinitely many worlds without allowing for a type of infinity within every world.

"Why?If spatial dimension are continuous, then you would generally expect the wavefunction to be spread over a dense infinite set of both basis vector both in the position basis and in the momentum basis."

The higher values do not realistically get realized and perhaps never at all. For fermions this means there can only be a finite anumber of them in a finite volume. Bosons can overlap but strange things will happen if too many of them overlap (they all interact with gravity). Of course infinity times something really small is still infinity, but that only goes to show that you will not have infinitely many worlds without some inherent infinities within our universe (infinite size of our universe, points with infinite density, particles with infinite energy), so one does not have to believe in infinite quantities to believe in many worlds

Why?If spatial dimension are continuous, then you would generally expect the wavefunction to be spread over a dense infinite set of both basis vector both in the position basis and in the momentum basis.

(If I understand correctly, under some technical conditions you could write the wavefunction w.r.t. a countable basis, but it would be still infinite unless the quantum system has finitely many degrees of freedom)

Aren't spatial dimensions essentially quantized for the purpose of counting alternative worlds because position accuracy is limited by energy and it seems particles above a certain high energy are so rare (or may even not exist at all) we have never seen one and aren't likely to even in the vicinity of a gamma ray burst (though of course this isn't a fixed grid like you would have when space itself was naturally quantized)?

Will Sawin already provided some excellent rebuttals, but we can look at it from less extreme angles. Is it war when your technology makes me starve to death (because you also crusaded against welfare and retraining vouchers) and I riot violently in return against the government you bought? It may not be an official war but it sure would be a bloody, possibly even genocidal affair.

"Are there different versions of many worlds providing different answers to this question?"

"This is to say, the infinity isn't countable but uncountable, like the number of geometric points said to lie on a line segment."

It depends on whether you believe states can change over infinitesimal timespans. It's very much possible time itself is quantized and even if it isn't it seems to be that there's, at least on average, a finite time required between state changes (a clue is that after the collapse of a wavefunction following a measurement it needs a measurable amount of time to become a spread out wavefunction again). So it seems to be that there are only a finite number of worlds in the many world interpretation IF the size of the universe is finite (which we just don't know).

I create a new set of technologies that allow me (or others I create) to make vastly more productive use of your land then you do. Claiming "manifest destiny", We kill you and take your resources, and create a legal system that allows us to do so safely.

Or:

I create a new set of technologies which leads to a further technological revolution until you are to us as pandas are to you. We take over your habitat and turn it into housing for trillions of ems. We make some attempt to preserve you in a zoo, but it is insufficient, and we are not willing to give up the resources to do better.

Or:

After several technological revolutions, you are to us as insects are to you. We destroy you with extreme prejudice.

Or:

After many technological revolutions, you are so slow-moving as to be seen as a mountain rather than a person. You are mined for natural resources.

----------

In creating ems and bringing forth a massive speedups in the rate of economic growth, all of these seem like real possibilities. "I don't know whether the tank I am building will kill you or not" is not a comforting defense.

If I send tanks into your nation to physically take control, that is war. If I invent a new kind of car likely to take away customers from your car factory, that isn't war.

In the debate, you described the possibility of an AI built by Eliezer reshaping the world in a period of 2 weeks as total war. The key feature of total war here being that when organizations believe it might happen they will fight viciously and brutally with other organizations to ensure it doesn't happen or happens in a way that's agreeable to then.

Do you think ems built by a corporation reshaping the world in a period of 6 years does not have that key feature?

The number of worlds in the many worlds interpretation is equal to the size of the phase space of the universe, which is finite if the universe is finite.

Are there different versions of many worlds providing different answers to this question? I find the following explanation on LW (strongly upvoted):

How many ink blots are in this picture ?For "many worlds", imagine that evolving in time, with fuzzy borders.The Many Worlds Interpretation doesn't imply a countable number of worlds that suddenly branch, it's more like a fuzzy continuum; talking of branching worlds and timelines is just a high-level abstraction that makes things easier to discuss.

This is to say, the infinity isn't countable but uncountable, like the number of geometric points said to lie on a line segment.

I've seen this enough on LW to feel confident it's the dominant interpretation there, but I have also seen, elsewhere, claims like yours, concluding there might be finite worlds.

By "metaphysical" I mean the theory's ontological commitments (as opposed to whether it formally invokes infinities). [I'm not a Quinean about the ontological commitments of theories, btw.]

I'm interested in what's involved in this disagreement, but also, note that my point here was about "infinite worlds" theories, not just MWI. For example, the Tegmark view that Yudkowsky endorses. Or the use of infinite worlds to create anthropic explanations.

But I get Jess Riedel's point that this may not be the place for detailed debate about the content of the book.

I understand the need not to have a deluge of posts on this tangent, but that problem has been ingeniously solved by Robin with his max. 3 visible posting rule. Am I missing something in Jess Riedel's complaint?

## Debate Is Now Book

"Doesn't that contradict your earlier claim:"

Well, a process that changes infinitely fast is a bit like an infinite density in time...

"I think what you're saying is that physical continuity (nonquantization of time or space) implies that infinitisimal time spans exist."

Yes, but even if infinitesimal time spans exist that does not have to mean any physical process works in infinitesimal time steps.

"I think assuming continuity implies actual infinitesimal time spans create Zeno's paradox"

Or processes just take an infinite number of infinitesimal steps at a time.

This reminds me of an Isaac Asimov short story where people have discovered how to travel to alternative universes. They reckon there are an infinite number of them so people start claiming alternative Earths for themselves and they expect to never encounter an alternative humanity. Than one day they do because the alternative Earths with humans on them are a fraction of all the worlds: the probability of meeting them is actually finite. Perhaps nature solves Zeno's paradox in a similar way.

you're definitely not going to get infinitely many worlds without allowing for a type of infinity within every world.

Doesn't that contradict your earlier claim:

"It depends on whether you believe states can change over infinitesimal timespans."

I think what you're saying is that physical continuity (nonquantization of time or space) implies that infinitisimal time spans exist. (I think assuming continuity implies actual infinitesimal time spans create Zeno's paradox - http://tinyurl.com/b9kn4tb )

Indeed you don't, infinite energy ranges or infinite densities also do the trick, it's just one way to get to an infinite number of possible worlds, but you're definitely not going to get infinitely many worlds without allowing for a type of infinity within every world.

You don't need infinitely many particles in order to have an infinitely dimensional Hilbert state-space

"Why?If spatial dimension are continuous, then you would generally expect the wavefunction to be spread over a dense infinite set of both basis vector both in the position basis and in the momentum basis."

The higher values do not realistically get realized and perhaps never at all. For fermions this means there can only be a finite anumber of them in a finite volume. Bosons can overlap but strange things will happen if too many of them overlap (they all interact with gravity). Of course infinity times something really small is still infinity, but that only goes to show that you will not have infinitely many worlds without some inherent infinities within our universe (infinite size of our universe, points with infinite density, particles with infinite energy), so one does not have to believe in infinite quantities to believe in many worlds

Why?If spatial dimension are continuous, then you would generally expect the wavefunction to be spread over a dense infinite set of both basis vector both in the position basis and in the momentum basis.

(If I understand correctly, under some technical conditions you could write the wavefunction w.r.t. a countable basis, but it would be still infinite unless the quantum system has finitely many degrees of freedom)

But differences in position and momentum can't be arbitrarily small, or if they can it's incredibly rare.

Due to deterministic chaos, arbitrarily small differences in position and momentum are amplified in relatively short amounts of time.

Aren't spatial dimensions essentially quantized for the purpose of counting alternative worlds because position accuracy is limited by energy and it seems particles above a certain high energy are so rare (or may even not exist at all) we have never seen one and aren't likely to even in the vicinity of a gamma ray burst (though of course this isn't a fixed grid like you would have when space itself was naturally quantized)?

It also depends on whether spatial dimensions are quantized, which is currently unknown.

Will Sawin already provided some excellent rebuttals, but we can look at it from less extreme angles. Is it war when your technology makes me starve to death (because you also crusaded against welfare and retraining vouchers) and I riot violently in return against the government you bought? It may not be an official war but it sure would be a bloody, possibly even genocidal affair.

"Are there different versions of many worlds providing different answers to this question?"

"This is to say, the infinity isn't countable but uncountable, like the number of geometric points said to lie on a line segment."

It depends on whether you believe states can change over infinitesimal timespans. It's very much possible time itself is quantized and even if it isn't it seems to be that there's, at least on average, a finite time required between state changes (a clue is that after the collapse of a wavefunction following a measurement it needs a measurable amount of time to become a spread out wavefunction again). So it seems to be that there are only a finite number of worlds in the many world interpretation IF the size of the universe is finite (which we just don't know).

Or:

I create a new set of technologies that allow me (or others I create) to make vastly more productive use of your land then you do. Claiming "manifest destiny", We kill you and take your resources, and create a legal system that allows us to do so safely.

Or:

I create a new set of technologies which leads to a further technological revolution until you are to us as pandas are to you. We take over your habitat and turn it into housing for trillions of ems. We make some attempt to preserve you in a zoo, but it is insufficient, and we are not willing to give up the resources to do better.

Or:

After several technological revolutions, you are to us as insects are to you. We destroy you with extreme prejudice.

Or:

After many technological revolutions, you are so slow-moving as to be seen as a mountain rather than a person. You are mined for natural resources.

----------

In creating ems and bringing forth a massive speedups in the rate of economic growth, all of these seem like real possibilities. "I don't know whether the tank I am building will kill you or not" is not a comforting defense.

If I send tanks into your nation to physically take control, that is war. If I invent a new kind of car likely to take away customers from your car factory, that isn't war.

In the debate, you described the possibility of an AI built by Eliezer reshaping the world in a period of 2 weeks as total war. The key feature of total war here being that when organizations believe it might happen they will fight viciously and brutally with other organizations to ensure it doesn't happen or happens in a way that's agreeable to then.

Do you think ems built by a corporation reshaping the world in a period of 6 years does not have that key feature?

The number of worlds in the many worlds interpretation is equal to the size of the phase space of the universe, which is finite if the universe is finite.

Are there different versions of many worlds providing different answers to this question? I find the following explanation on LW (strongly upvoted):

How many ink blots are in this picture ?For "many worlds", imagine that evolving in time, with fuzzy borders.The Many Worlds Interpretation doesn't imply a countable number of worlds that suddenly branch, it's more like a fuzzy continuum; talking of branching worlds and timelines is just a high-level abstraction that makes things easier to discuss.

http://lesswrong.com/lw/8ue...

This is to say, the infinity isn't countable but uncountable, like the number of geometric points said to lie on a line segment.

I've seen this enough on LW to feel confident it's the dominant interpretation there, but I have also seen, elsewhere, claims like yours, concluding there might be finite worlds.

By "metaphysical" I mean the theory's ontological commitments (as opposed to whether it formally invokes infinities). [I'm not a Quinean about the ontological commitments of theories, btw.]

I'm interested in what's involved in this disagreement, but also, note that my point here was about "infinite worlds" theories, not just MWI. For example, the Tegmark view that Yudkowsky endorses. Or the use of infinite worlds to create anthropic explanations.

But I get Jess Riedel's point that this may not be the place for detailed debate about the content of the book.

I understand the need not to have a deluge of posts on this tangent, but that problem has been ingeniously solved by Robin with his max. 3 visible posting rule. Am I missing something in Jess Riedel's complaint?