Here Be Dragons

In his new book Here Be Dragons: Science, technology, and the future of humanity, Olle Haggstrom mostly discusses abstract and philosophical issues. But at one point in the book he engages the more specific forecasts I discuss in my upcoming book. So let me quote him and offer a few responses:

Once successful whole-brain emulation has been accomplished, it might not be long before it becomes widely available and widely-used. This bring us to question (4) – what will society be like when uploading is widely available? Most advocates of an uploaded posthuman existence, such as Kurzweil and Goertzel, point at the unlimited possibilities for an unimaginably (to us) riche and wonderful life in ditto virtual realities. One researcher stands out from the rest in actually applying economic theory and social science to attempt to sketch how a society of uploads will turn out is the American economist Robin Hanson, beginning in a 1994 paper, continuing with a series of posts on his extraordinary blog Overcoming Bias, and summarizing his findings (so far) in a chapter in Intelligence Unbound and in an upcoming book.

Two basic assumptions for Hanson’s social theory of uploads are
(i) that whole-brain emulation is achieved mostly by brute force, with relatively little scientific understanding of how thoughts and other high-level phenomena supervene on the lower-level processes that are simulated, and
(ii) that current trends of hardware costs decreasing at a fast exponential rate will continue (if not indefinitely then at least far into the era he describes).

Actually, I just need to assume that at some point the hardware cost is low enough to make uploads substantially cheaper than human workers. I don’t need to make assumptions about rates at which hardware costs fall.

Assumption (i) prevents us from boosting the emulated minds to superhuman intelligence levels, other than in terms of speed, by transferring the mot faster hardware. Assumption (ii) opens up the possibility for quickly populating the world with billions and then trillions of uploaded minds, which is in fact what Hanson predicts will happen. ..

Actually, population increases quickly mainly because factories can crank out an amount of hardware equal to their own economic value in a short time – months or less.

Decreases in hardware costs will push down wages. .. This will send society to the classical Malthusian trap in which population will grown until it is hit by starvation (uploaded minds will not need food, of course, but things like energy, CPU time and disk space). ..

There are many exotica in Hanson’s future. One is that uploads can fun on different hardware and thus at different speeds, depending on circumstances. .. Even more exotic is the idea that most work will be done by short-lived so-called spurs, copied from a template upload to work for, say, a few hours and then be terminated (i.e., die). .. Will they not revolt? The question has been asked, but Hanson maintains that “when life is cheap, death is cheap.”

First, spurs could retire to a much slower speed instead of ending. Second, just before an em considers whether to split off a spur copy for a task, that em can ask itself if it would be willing to do that assigned task if it found itself a few seconds later to be the spur. Ems should quickly learn to reliable estimate their own willingness, so they just won’t split off spurs if they estimated a high chance that the spur would become troublesome. Maybe today we find it hard to estimate such things, but they’d know their world well so it would an easy question for them. So I just can’t see spur rebellion as a big practical problem, any more than we have a big problem planning to go to work for the day and then suddenly going to the movies instead.

The future outlined in Hanson’s theory of uploaded minds may seem dystopian .. but Hanson does not accept this label, and his main retorts seem to be twofold. First, population numbers will be huge, which is good if we accept that the value of a future should be measured .. by the total amount of well-being, which in a huge population can be very large even if each individual has only a modest positive level of well-being. Second, the trillions of short-lived uploaded minds working hard for their subsistence right near starvation level can be made to enjoy themselves, e.g., by cheap artificial stimulation of their pleasure center.

I don’t think I’ve ever talked about “cheap artificial stimulation of their pleasure center.” I instead say that most ems work and leisure in virtual worlds of spectacular quality, and that ems need never experience hunger, disease, or intense pain, nor ever see, hear, feel, or taste grime or anything ugly or disgusting. Yes they’d work most of the time but their jobs would be mentally challenging, they’d be selected for being very good at their jobs, and people can find deep fulfillment in such modes. We are very culturally plastic, and em culture would promote finding value and fulfillment in typical em lives. In addition, I estimate that most humans who have ever lived have had lives worth living, in part because of this cultural plasticity.

Then there’s the issue of whether and to what extent we should view Hanson’s analysis as a trustworthy prediction of what will actually happen. A healthy load of skepticism seems appropriate. .. It also seems that he works so far outside of the comfort zones of where economic theory has been tested empirically, and uses so many explicit and implicit assumptions that are open to questioning, that his scenarios need to be taken with a grain of salt (or a full bushel).

You could say this about any theoretical analysis of anything not yet seen. All theory requires you to make assumptions, and all assumptions are open to questioning. Perhaps my case is worse than others, but the above certainly doesn’t show that to be the case.

One obvious issue to consider is whether society following a breakthrough in the technology will be better or worse than society without such a breakthrough. The utopias hinted at by, e.g., Kurzweil and Goertzel seem pretty good, whereas Hanson’s Malthusian scenario looks rather less appealing.

But Kurzweil and Goertzel offer inspiring visions, not hard-headed social science analysis. Of course that will sound better.