Tag Archives: Future

The One Ruler Obsession

I often teach undergraduate law & economics. Sometimes the first paper I assign is to suggest property rules to deal with conflicts regarding asteroids, orbits, and sunlight in the solar system, in the future when there’s substantial activity out there. This feels to students like a complex different situation, and in fact few understand even the basic issues.

Given just two pages to make their case, a large fraction of students (~1/3?) express fear that one person or organization will take over the entire solar system, unless property rules are designed to explicitly prevent that. And a similar fraction suggest the “property rule” of having a single government agency answer all questions. Whatever question or dispute you have, fill out a form, and the agency will decide.

Yet in my lectures I talk a lot about concepts and issues of property rights, but never mention government agency issues or scenarios, nor the scenario of one power taking over everything. And econ undergrads at my school are famous for being relatively libertarian.

I conclude that most people have a strong innate fear of power concentrations, and yet also see the creation of a single central power as an attractive general solution to complicated problems. I’ve seen the same sort of thing with a great many futuristic tech and policy issues. Whatever the question, if it seems complicated, most people are concerned about inequality, especially that it might be taken to the max, and yet they also like the idea of creating a central government-like power to deal with it.

I’ve certainly seen this in concerns about future rampaging robots (= “AI risk”). Many, perhaps most, people express concerns that one AI could take over everything, and many also like the “solution” of one good AI taking over everything.

I recently came across similar reasoning by Frederick Engels back in 1844, in his Outlines of a Critique of Political Economy. Having seen the early industrial revolution, not understanding it well, but fearing where it might lead, Engels claims that the natural outcome is extreme concentration of power. And his solution is to create a different central power (e.g., communism). Of course while there was some increase in inequality and concentration, it wasn’t remotely as bad as Engels feared, except where his words helped to inspire the creation of such concentration. Here is Engels:

Thus, competition sets capital against capital, labour against labour, landed property against landed property; and likewise each of these elements against the other two. In the struggle the stronger wins; and in order to predict the outcome of the struggle, we shall have to investigate the strength of the contestants. First of all, labour is weaker than either landed property or capital, for the worker must work to live, whilst the landowner can live on his rent, and the capitalist on his interest, or, if the need arises, on his capital or on capitalised property in land. The result is that only the very barest necessities, the mere means of subsistence, fall to the lot of labour; whilst the largest part of the products is shared between capital and landed property. Moreover, the stronger worker drives the weaker out of the market, just as larger capital drives out smaller capital, and larger landed property drives out smaller landed property. Practice confirms this conclusion. The advantages which the larger manufacturer and merchant enjoy over the smaller, and the big landowner over the owner of a single acre, are well known. The result is that already under ordinary conditions, in accordance with the law of the stronger, large capital and large landed property swallow small capital and small landed property – i.e., centralisation of property. In crises of trade and agriculture, this centralisation proceeds much more rapidly.

In general large property increases much more rapidly than small property, since a much smaller portion is deducted from its proceeds as property-expenses. This law of the centralisation of private property is as immanent in private property as all the others. The middle classes must increasingly disappear until the world is divided into millionaires and paupers, into large landowners and poor farm labourers. All the laws, all the dividing of landed property, all the possible splitting-up of capital, are of no avail: this result must and will come, unless it is anticipated by a total transformation of social conditions, a fusion of opposed interests, an abolition of private property.

Free competition, the keyword of our present-day economists, is an impossibility. Monopoly at least intended to protect the consumer against fraud, even if it could not in fact do so. The abolition of monopoly, however, opens the door wide to fraud. You say that competition carries with it the remedy for fraud, since no one will buy bad articles. But that means that everyone has to be an expert in every article, which is impossible. Hence the necessity for monopoly, which many articles in fact reveal. Pharmacies, etc., must have a monopoly. And the most important article – money – requires a monopoly most of all. Whenever the circulating medium has ceased to be a state monopoly it has invariably produced a trade crisis; and the English economists, Dr. Wade among them, do concede in this case the necessity for monopoly. But monopoly is no protection against counterfeit money. One can take one’s stand on either side of the question: the one is as difficult as the other. Monopoly produces free competition, and the latter, in turn, produces monopoly. Therefore both must fall, and these difficulties must be resolved through the transcendence of the principle which gives rise to them. (more)

GD Star Rating
loading...
Tagged as: ,

How Big Future Change?

The world has seen a lot of very big changes over the last few centuries. Many of these changes seem so large, in fact, that it is hard to see how changes over the next few centuries could be remotely as large. For example, many “big swing” parameters have moved from one extreme to the other, changing by more than half of the total range possible for that parameter. So the only way future changes could be as large in such a parameter is if it completely reversed direction to move back to the opposite extreme.

For example, once only a small percentage of people lived in cities; now more than half do. Once only a few nations were democratic, now more than half are. Once many people were slaves, now there are very few slaves. Once people worked nearly as many hours a week as possible, now they work less than half of their waking hours. Once nations were frequently at war, now war is rare. Once lifespans were near 30 years, now they are near 80, and some say 120 is the max possible. Once few people could read, now most can. Once genders and races were treated quite unequally, now treatment is more equal than unequal. Once engines and solar cells had low efficiency, now efficiency is half or more of the theoretical maximum. And so on.

If these big-swing parameters encompassed most of what we cared about in change, and if it is in fact implausible for such parameters to reverse back to their opposite extremes, then the conclusion seems inescapable: future change must be less than past change.

But pause to ask: how sure can we be that these big swing parameters encompass a large fraction of what matters within what can change? And notice a big selection effect: even when rates of change are constant overall, the particular parameters that happened to change the most in the recent past will in general not be the ones that change the most in the near future. So for those big past changing params future change will be less, even though overall rates of change stay steady. Maybe we spend so much time focusing on the parameters that have recently changed most, that we forget how many other parameters remain which are available to change in the future.

My book Age of Em might be taken as a demonstration that big future change remain possible. And we might also test this selection effect via a historical analysis. We might, for example, look at params that changed the most from the year 500 to the year 1000, at least as people in the year 1000 would have seen them, and then ask if those particular parameters changed more or less during the period from 1000 to 1500. Repeat for many different times and places.

GD Star Rating
loading...
Tagged as: ,

Reply to Christiano on AI Risk

Paul Christiano was one of those who encouraged me to respond to non-foom AI risk concerns. Here I respond to two of his posts he directed me to. The first one says we should worry about the following scenario:

Imagine using [reinforcement learning] to implement a decentralized autonomous organization (DAO) which maximizes its profit. .. to outcompete human organizations at a wide range of tasks — producing and selling cheaper widgets, but also influencing government policy, extorting/manipulating other actors, and so on.

The shareholders of such a DAO may be able to capture the value it creates as long as they are able to retain effective control over its computing hardware / reward signal. Similarly, as long as such DAOs are weak enough to be effectively governed by existing laws and institutions, they are likely to benefit humanity even if they reinvest all of their profits.

But as AI improves, these DAOs would become much more powerful than their human owners or law enforcement. And we have no ready way to use a prosaic AGI to actually represent the shareholder’s interests, or to govern a world dominated by superhuman DAOs. In general, we have no way to use RL to actually interpret and implement human wishes, rather than to optimize some concrete and easily-calculated reward signal. I feel pessimistic about human prospects in such a world. (more)

In a typical non-foom world, if one DAO has advanced abilities, then most other organizations, including government and the law, have similar abilities. So such DAOs shouldn’t find it much easier to evade contracts or regulation than do organizations today. Thus humans can be okay if law and government still respect human property rights or political representation. Sure it might be hard to trust such a DAO to manage your charity, if you don’t trust it to judge who is in most need. But you might trust it much to give you financial returns on your financial investments in it.

Paul Christiano’s second post suggests that the arrival of AI arrives will forever lock in the distribution of patient values at that time:

The distribution of wealth in the world 1000 years ago appears to have had a relatively small effect—or more precisely an unpredictable effect, whose expected value was small ex ante—on the world of today. I think there is a good chance that AI will fundamentally change this dynamic, and that the distribution of resources shortly after the arrival of human-level AI may have very long-lasting consequences. ..

Whichever values were most influential at one time would remain most influential (in expectation) across all future times. .. The great majority of resources are held by extremely patient values. .. The development of machine intelligence may move the world much closer to this naïve model. .. [Because] the values of machine intelligences can (probably, eventually) be directly determined by their owners or predecessors. .. it may simply be possible to design a machine intelligence who exactly shares their predecessor’s values and who can serve as a manager. .. the arrival of machine intelligence may lead to a substantial crystallization of influence .. an event with long-lasting consequences. (more)

That is, Christiano says future AI won’t have problems preserving its values over time, nor need it pay agency costs to manage subsystems. Relatedly, Christiano elsewhere claims that future AI systems won’t have problems with design entrenchment:

Over the next 100 years greatly exceeds total output over all of history. I agree that coordination is hard, but even spending a small fraction of current effort on exploring novel redesigns would be enough to quickly catch up with stuff designed in the past.

A related claim, that Christiano supports to some degree, is that future AI are smart enough to avoid suffers from coordination failures. They may even use “acasual trade” to coordinate when physical interaction of any sort is impossible!

In our world, more competent social and technical systems tend to be larger and more complex, and such systems tend to suffer more (in % cost terms) from issues of design entrenchment, coordination failures, agency costs, and preserving values over time. In larger complex systems, it becomes harder to isolate small parts that encode “values”; a great many diverse parts end up influencing what such systems do in any given situation.

Yet Christiano expects the opposite for future AI; why? I fear his expectations result more from far view idealizations than from observed trends in real systems. In general, we see things far away in less detail, and draw inferences about them more from top level features and analogies than from internal detail. Yet even though we know less about such things, we are more confident in our inferences! The claims above seem to follow from the simple abstract description that future AI is “very smart”, and thus better in every imaginable way. This is reminiscent of medieval analysis that drew so many conclusions about God (including his existence) from the “fact” that he is “perfect.”

But even if values will lock in when AI arrives, and then stay locked, that still doesn’t justify great efforts to study AI control today, at least relative to the other options of improving our control mechanisms in general, or saving resources now to spend later, either on studying AI control problems when we know more about AI, or just to buy influence over the future when that comes up for sale.

GD Star Rating
loading...
Tagged as: , , ,

An Outside View of AI Control

I’ve written much on my skepticism of local AI foom (= intelligence explosion). Recently I said that foom offers the main justification I understand for AI risk efforts now, as well as being the main choice of my Twitter followers in a survey. It was the main argument offered by Eliezer Yudkowsky in our debates here at this blog, by Nick Bostrom in his book Superintelligence, and by Max Tegmark in his recent book Life 3.0 (though he denied so in his reply here).

However, some privately complained to me that I haven’t addressed those with non-foom-based AI concerns. So in this post I’ll consider AI control in the context of a prototypical non-em non-foom mostly-peaceful outside-view AI scenario. In a future post, I’ll try to connect this to specific posts by others on AI risk.

An AI scenario is where software does most all jobs; humans may work for fun, but they add little value. In a non-em scenario, ems are never feasible. As foom scenarios are driven by AI innovations that are very lumpy in time and organization, in non-foom scenarios innovation lumpiness is distributed more like it is in our world. In a mostly-peaceful scenario, peaceful technologies of production matter much more than do technologies of war and theft. And as an outside view guesses that future events are like similar past events, I’ll relate future AI control problems to similar past problems. Continue reading "An Outside View of AI Control" »

GD Star Rating
loading...
Tagged as: , ,

Humans Cells In Multicellular Future Minds?

In general, adaptive systems vary along an axis from general to specific. A more general system works better (either directly or after further adaptation) in a wider range of environments, and also with a wider range of other adapting systems. It does this in part via having more useful modularity and abstraction. In contrast, a more specific system adapts to a narrower range of specific environments and other subsystems.

Systems that we humans consciously design tend to be more general, i.e., less context dependent, relative to the “organic” systems that they often replace. For example, compare grid-like city street plans to locally evolved city streets, national retail outlets to locally arising stores and restaurants, traditional to permaculture farms, hotel rooms to private homes, big formal firms to small informal teams, uniforms to individually-chosen clothes, and refactored to un-refactored software. The first entity in each pair tends to more easily scale and to match more environments, while the second in each pair tends to be adapted in more detail to particular local conditions. Continue reading "Humans Cells In Multicellular Future Minds?" »

GD Star Rating
loading...
Tagged as: ,

Tegmark’s Book of Foom

Max Tegmark says his new book, Life 3.0, is about what happens when life can design not just its software, as humans have done in Life 2.0, but also its hardware:

Life 1.0 (biological stage) evolves its hardware and software
Life 2.0 (cultural stage) evolves its hardware, designs much of its software
Life 3.0 (technological stage): designs its hardware and software ..
Many AI researchers think that Life 3.0 may arrive during the coming century, perhaps even during our lifetime, spawned by progress in AI. What will happen, and what will this mean for us? That’s the topic of this book. (29-30)

Actually, its not. The book says little about redesigning hardware. While it says interesting things on many topics, its core is on a future “singularity” where AI systems quickly redesign their own software. (A scenario sometimes called “foom”.)

The book starts out with a 19 page fictional “scenario where humans use superintelligence to take over the world.” A small team, apparently seen as unthreatening by the world, somehow knows how to “launch” a “recursive self-improvement” in a system focused on “one particular task: programming AI Systems.” While initially “subhuman”, within five hours it redesigns its software four times and becomes superhuman at its core task, and so “could also teach itself all other humans skills.”

After five more hours and redesigns it can make money by doing half of the tasks at Amazon Mechanical Turk acceptably well. And it does this without having access to vast amounts of hardware or to large datasets of previous performance on such tasks. Within three days it can read and write like humans, and create world class animated movies to make more money. Over the next few months it goes on to take over the news media, education, world opinion, and then the world. It could have taken over much faster, except that its human controllers were careful to maintain control. During this time, no other team on Earth is remotely close to being able to do this.

Tegmark later explains: Continue reading "Tegmark’s Book of Foom" »

GD Star Rating
loading...
Tagged as: , ,

Can Human-Like Software Win?

Many, perhaps most, think it obvious that computer-like systems will eventually be more productive than human-like systems in most all jobs. So they focus on how humans might maintain control, even after this transition. But this eventuality is less obvious than it seems, depending on what exactly one means by “human-like” or “computer-like” systems. Let me explain.

Today the software that sits in human brains is stuck in human brain hardware, while the other kinds of software that we write (or train) sit in the artificial hardware that we make. And this artificial hardware has been improving rapidly far more rapidly than has human brain hardware. Partly as a result of this, systems of artificial software and hardware have been improving rapidly compared to human brain systems.

But eventually we will find a way to transfer the software from human brains into artificial hardware. Ems are one way to do this, as a relatively direct port. But other transfer mechanics may be developed.

Once human brain software is in the same sort of artificial computing hardware as all the other software, then the relative productivity of different software categories comes down to a question of quality: which categories of software tend to be more productive on which tasks?

Of course there will many different variations available within each category, to match to different problems. And the overall productivity of each category will depend both on previous efforts to develop and improve software in that category, and also on previous investments in other systems to match and complement that software. For example, familiar artificial software will gain because we have spent longer working to match it to familiar artificial hardware, while human software will gain from being well matched to complex existing social systems, such as language, firms, law, and government.

People give many arguments for why they expect human-like software to mostly lose this future competition, even when it has access to the same hardware. For example, they say that other software could lack human biases and also scale better, have more reliable memory, communicate better over wider scopes, be easier to understand, have easier meta-control and self-modification, and be based more directly on formal abstract theories of learning, decision, computation, and organization.

Now consider two informal polls I recently gave my twitter followers:

Surprisingly, at least to me, the main reason that people expect human-like software to lose is that they mostly expect whole new categories of software to appear, categories quite different from both the software in the human brain and also all the many kinds of software with which we are now familiar. If it comes down to a contest between human-like and familiar software categories, only a quarter of them expect human-like to lose big.

The reason I find this surprising is that all of the reasons that I’ve seen given for why human-like software could be at a disadvantage seem to apply just as well to familiar categories of software. In addition, a new category must start with the disadvantages of having less previous investment in that category and in matching other systems to it. That is, none of these are reasons to expect imagined new categories of software to beat familiar artificial software, and yet people offer them as reasons to think whole new much more powerful categories will appear and win.

I conclude that people don’t mostly use specific reasons to conclude that human-like software will lose, once it can be moved to artificial hardware. Instead they just have a general belief that the space of possible software is huge and contains many new categories to discover. This just seems to be the generic belief that competition and innovation will eventually produce a lot of change. Its not that human-like software has any overall competitive disadvantage compared to concrete known competitors; it is at least as likely to have winning descendants as any such competitors. Its just that our descendants are likely to change a lot as they evolve over time. Which seems to me a very different story than the humans-are-sure-to-lose story we usually hear.

GD Star Rating
loading...
Tagged as: , ,

A Post-Em-Era Hint

A few months ago I noticed a pattern across the past eras of forager, farmer industry: each era has a major cycle (ice ages, empires rise & fall, business cycle) with a period of about one third of that era’s doubling time. So I tentatively suggested that a em future might also have a major cycle of roughly one third of its doubling time. If that economic doubling time is about a month, the em major cycle period might be about a week.

Now I report another pattern, to be treated similarly. In roughly the middle of each past era, a pair of major innovations in calculating and communicating appeared, and gradually went from barely existing to having big social impacts.

  • Forager: At unknown periods during the roughly two million year forager era, humanoids evolved reasoning and language. That is, we became able to think about and say many complex things to each other, including our reasons for and against claims.
  • Farmer: While the farming era lasted roughly 7 to 10 millennia, the first known writing was 5 millennia ago, and the first known math textbooks 4 millennia ago. About 2.5 millennia ago writing became widespread enough to induce major religious changes worldwide.
  • Industry: While the industry era has lasted roughly 16 to 24 decades, depending on how you count, the telegraph was developed 18 decades ago, and the wholesale switch from mechanical to digital electronic communication happened 4 to 6 decades ago. The idea of the computer was described 20 decades ago, the first digital computer was made 7 decades ago, and computers became widespread roughly 3 decades ago.

Note that innovations in calculation and communication were not independent, but instead intertwined with and enabled each other. Note also that these innovations did not change the growth rate of the world economy at the time; each era continued doubling at the same rate as before. But these innovations still seem essential to enabling the following era. It is hard to imagine farming before language and reasoning, nor industry before math and writing, nor ems before digital computers and communication.

This pattern weakly suggests that another pair of key innovations in calculation and communication may appear and then grow in importance across a wide middle of the em era. This era may only last a year or two in objective time, though typical ems may experience millennia during this time.

This innovation pair would be interdependent, not change the growth rate, and perhaps enable a new era to follow. I can think of two plausible candidates:

  1. Ems might discover a better language for expressing and manipulating something like brain states. This could help ems to share their thoughts and use auxiliary hardware to help calculate useful thoughts.
  2. Ems might develop analogues to combinatorial prediction markets, and thus better share beliefs and aggregate information on a wide range of topics.

(Or maybe the innovation produces some combination of these.) Again, these are crude speculations based on a weak inference from a rough pattern in only three data points. But even so, they give us a vague hint about what an age after ems might look like. And such hints are actually pretty hard to find.

GD Star Rating
loading...
Tagged as: , , ,

A Tangled Task Future

Imagine that you want to untangle a pile of cables. It wasn’t tangled on purpose; tangling just resulted naturally from how these cables were used. You’d probably look for the least tangled cable in the least tangled part of the pile, and start to work there. In this post I will argue that, in a nutshell, this is how we are slowly automating our world of work: we are un- and re-tangling it.

This has many implications, including for the long-term future of human-like creatures in a competitive world. But first we have a bit of explaining to do. Continue reading "A Tangled Task Future" »

GD Star Rating
loading...
Tagged as: , ,

Mormon Transhumanists

A standard trope of science fiction has religious groups using violence to stop a new technology. Perhaps because of this, many are surprised by the existence of religious transhumanists. Saturday I gave a keynote talk on Age of Em at the Mormon Transhumanist Association (MTA) annual conference, and had a chance to study such folks in more detail. And I should say right off the top that this MTA audience, compared to other audiences, had notably fewer morality or religious related objections to my em scenario.

I’m not surprised by the existence of religious tech futurists. Overall, the major world religions have been quite successful in adapting to the many social changes since most of them first appeared many millennia ago. Also, the main predictor of interest in tech futurism and science fiction is an interest in science and technology, and religious folks are not underrepresented there. Even so, you might ask what your favorite theories of religion predict about how MTA folk would differ from other transhumanists.

The most obvious difference I saw is that MTA does community very well, with good organization, little shirking, and lots of polite, respectful, and friendly interaction. This makes sense. Mormons in general have strong community norms, and one of the main functions of religion is to build strong communities. Mormonism is a relatively high commitment religion, and those tend to promote stronger bonds.

Though I did not anticipate it, a predictable consequence of this is that MTA is more of a transhuman take on Mormonism than a Mormon take on transhumanism. On reflection, this reveals an interesting way that long-lived groups with dogmas retain and co-op smart intellectuals. Let me explain.

One standard sales technique is to try to get your mark to spend lots of time considering your product. This is a reason why salespeople often seem so slow and chatty. The more time you spend considering their product, the longer that you will estimate it will take to consider other products, and the more likely you are to quit searching and take their product.

Similarly, religions often expose children to a mass of details, as in religious stories. Smart children can be especially engaged by these details because they like to show off their ability to remember and understand detail. Later on, such people can show off their ability to interpret these details in many ways, and to identify awkward and conflicting elements.

Even if the conflicts they find are so severe as to reasonably call into question the entire thing, by that time such people have invested so much in learning details of their religion that they’d lose a lot of ability to show off if they just left and never talked about it again. Some become vocally against their old religion, which lets them keep talking and showing off about it. But even in opposition, they are still then mostly defined by that religion.

I didn’t meet any MTA who took Mormon claims on miraculous historical events literally. They seemed well informed on science and tech and willing to apply typical engineering and science standards to such things. Even so, MTA folks are so focused on their own Mormon world that they tend to be less interested in asking how Mormons could anticipate and prepare for future changes, and more interested in how future/sci/tech themes could reframe and interpret key Mormon theological debates and claims. In practice their strong desire to remain Mormons in good standing means that they mostly accept practical church authority, including the many ways that the church hides the awkward and conflicting elements of its religions stories and dogma.

For example, MTA folks exploring a “new god argument” seek scenarios wherein we might live in a simulation that resonate with Mormon claims of a universe full of life and gods. While these folks aren’t indifferent to the relative plausibility of hypotheses, this sort of exercise is quite different from just asking what sort of simulations would be most likely if we in fact did live in a simulation.

I’ve said that we today live in an unprecedented dreamtime of unadaptive behavior, a dream from which some will eventually awake. Religious folks in general tend to be better positioned to awake sooner, as they have stronger communities, more self-control, and higher fertility. But even if the trope applies far more in fiction than in reality, it remains possible that Mormon religious orthodoxy could interfere with Mormons adapting to the future.

MTA could help to deal with such problems by becoming trusted guides to the future for other Mormons. To fill that role, they would of course need to show enough interest in Mormon theology to convince the others that they are good Mormons. But they would also need to pay more attention to just studying the future regardless of its relevance to Mormon theology. Look at what is possible, what is likely, and the consequences of various actions. For their sakes, I hope that they can make this adjustment.

By the way, we can talk similarly about libertarians who focus on criticizing government regulation and redistribution. The more one studies the details of government actions, showing off via knowing more such detail, then even if one mostly criticizes such actions, still one’s thinking becomes mostly defined by government. To avoid this outcome, focus more on thinking about what non-government organizations should do and how. It isn’t enough to say “without government, the market will do it.” Become part of a market that does things.

GD Star Rating
loading...
Tagged as: , ,