Overcoming Bias

Abstract/Distant Future Bias

The latest Science has a psych article saying we think of distant stuff more abstractly, and vice versa.  "The brain is hierarchically organized with higher points in the cortical hierarchy representing increasingly more abstract aspects of stimuli"; activating a region makes nearby activations more likely.  This has stunning implications for our biases about the future. 

All of these bring each other more to mind: here, now, me, us; trend-deviating likely real local events; concrete, context-dependent, unstructured, detailed, goal-irrelevant incidental features; feasible safe acts; secondary local concerns; socially close folks with unstable traits. 

Conversely, all these bring each other more to mind: there, then, them; trend-following unlikely hypothetical global events; abstract, schematic, context-freer, core, coarse, goal-related features; desirable risk-taking acts, central global symbolic concerns, confident predictions, polarized evaluations, socially distant people with stable traits. 

Since these things mostly just cannot go together in reality, this must bias our thinking both about now and about distant futures.  When "in the moment," we focus on ourselves and in-our-face details, feel "one with" what we see and close to quirky folks nearby, see much as uncertain, and safely act to achieve momentary desires given what seems the most likely current situation.  Kinda like smoking weed.

Regarding distant futures, however, we'll be too confident, focus too much on unlikely global events, rely too much on trends, theories, and loose abstractions, while neglecting details and variation.  We'll assume the main events take place far away (e.g., space), and uniformly across large regions.  We'll focus on untrustworthy consistently-behaving globally-organized social-others.  And we'll neglect feasibility, taking chances to achieve core grand symbolic values, rather than ordinary muddled values.  Sound familiar?

...Recursion, Magic

Followup to:  Cascades, Cycles, Insight...

...4, 5 sources of discontinuity.

Recursion is probably the most difficult part of this topic.  We have historical records aplenty of cascades, even if untangling the causality is difficult.  Cycles of reinvestment are the heartbeat of the modern economy.  An insight that makes a hard problem easy, is something that I hope you've experienced at least once in your life...

But we don't have a whole lot of experience redesigning our own neural circuitry.

We have these wonderful things called "optimizing compilers".  A compiler translates programs in a high-level language, into machine code (though these days it's often a virtual machine).  An "optimizing compiler", obviously, is one that improves the program as it goes.

So why not write an optimizing compiler in its own language, and then run it on itself?  And then use the resulting optimized optimizing compiler, to recompile itself yet again, thus producing an even more optimized optimizing compiler -

Halt!  Stop!  Hold on just a minute!  An optimizing compiler is not supposed to change the logic of a program - the input/output relations.  An optimizing compiler is only supposed to produce code that does the same thing, only faster.  A compiler isn't remotely near understanding what the program is doing and why, so it can't presume to construct a better input/output function.  We just presume that the programmer wants a fixed input/output function computed as fast as possible, using as little memory as possible.

So if you run an optimizing compiler on its own source code, and then use the product to do the same again, it should produce the same output on both occasions - at most, the first-order product will run faster than the original compiler.

If we want a computer program that experiences cascades of self-improvement, the path of the optimizing compiler does not lead there - the "improvements" that the optimizing compiler makes upon itself, do not improve its ability to improve itself.

When Life Is Cheap, Death Is Cheap

Carl, thank you for thoughtfully engaging my whole brain emulation scenario.  This is my response. 

Hunters couldn't see how exactly a farming life could work, nor could farmers see how exactly an industry life could work.  In both cases the new life initially seemed immoral and repugnant to those steeped in prior ways.  But even though prior culture/laws typically resisted and discouraged the new way, the few groups which adopted it won so big others were eventually converted or displaced.

Carl considers my scenario of a world of near-subsistence-income ems in a software-like labor market, where millions of cheap copies are made of a each expensively trained em, and then later evicted from their bodies when their training becomes obsolete.  Carl doesn't see how this could work: 

The Alices now know that Google will shortly evict them, the genocide of a tightly knit group of millions: will they peacefully comply with that procedure? Or will they use politics, violence and any means necessary to get capital from capital-holders so that they can continue to exist? If they seek allies, the many other ems who expect to be driven out of existence by competitive niche exclusion might be interested in cooperating with them. ... In order ... that biological humans could retain their wealth as capital-holders in his scenario, ems must be obedient and controllable enough that whole lineages will regularly submit to genocide, even though the overwhelming majority of the population expects the same thing to happen to it soon. But if such control is feasible, then a controlled em population being used to aggressively create a global singleton is also feasible.

I see pathologically-obedient personalities neither as required for my scenario, nor as clearly leading to a totalitarian world regime.

Cascades, Cycles, Insight...

Followup to:  Surprised by Brains

Five sources of discontinuity:  1, 2, and 3...

Cascades are when one thing leads to another.  Human brains are effectively discontinuous with chimpanzee brains due to a whole bag of design improvements, even though they and we share 95% genetic material and only a few million years have elapsed since the branch.  Why this whole series of improvements in us, relative to chimpanzees?  Why haven't some of the same improvements occurred in other primates?

Well, this is not a question on which one may speak with authority (so far as I know).  But I would venture an unoriginal guess that, in the hominid line, one thing led to another.

The chimp-level task of modeling others, in the hominid line, led to improved self-modeling which supported recursion which enabled language which birthed politics that increased the selection pressure for outwitting which led to sexual selection on wittiness...

...or something.  It's hard to tell by looking at the fossil record what happened in what order and why.  The point being that it wasn't one optimization that pushed humans ahead of chimps, but rather a cascade of optimizations that, in Pan, never got started.

We fell up the stairs, you might say.  It's not that the first stair ends the world, but if you fall up one stair, you're more likely to fall up the second, the third, the fourth...

"Evicting" brain emulations

Follow up to: Brain Emulation and Hard Takeoff

Suppose that Robin's Crack of a Future Dawn scenario occurs: whole brain emulations ('ems') are developed, diverse producers create ems of many different human brains, which are reproduced extensively until the marginal productivity of em labor approaches marginal cost, i.e. Malthusian near-subsistence wages. Ems that hold capital could use it to increase their wealth by investing, e.g. by creating improved ems and collecting the fruits of their increased productivity, by investing in hardware to rent to ems, or otherwise. However, an em would not be able to earn higher returns on its capital than any other investor, and ems with no capital would not be able to earn more than subsistence (including rental or licensing payments). In Robin's preferred scenario, free ems would borrow or rent bodies, devoting their wages to rental costs, and would be subject to "eviction" or "repossession" for nonpayment.

In this intensely competitive environment, even small differences in productivity between em templates will result in great differences in market share, as an em template with higher productivity can outbid less productive templates for scarce hardware resources in the rental market, resulting in their "eviction" until the new template fully supplants them in the labor market. Initially, the flow of more productive templates and competitive niche exclusion might be driven by the scanning of additional brains with varying skills, abilities, temperament, and values, but later on em education and changes in productive skill profiles would matter more.

Surprised by Brains

Followup to:  Life's Story Continues

Imagine two agents who've never seen an intelligence - including, somehow, themselves - but who've seen the rest of the universe up until now, arguing about what these newfangled "humans" with their "language" might be able to do...

Believer:  Previously, evolution has taken hundreds of thousands of years to create new complex adaptations with many working parts.  I believe that, thanks to brains and language, we may see a new era, an era of intelligent design. In this era, complex causal systems - with many interdependent parts that collectively serve a definite function - will be created by the cumulative work of many brains building upon each others' efforts.

Skeptic:  I see - you think that brains might have something like a 50% speed advantage over natural selection?  So it might take a while for brains to catch up, but after another eight billion years, brains will be in the lead.  But this planet's Sun will swell up by then, so -

Believer:  Thirty percent?  I was thinking more like three orders of magnitude. With thousands of brains working together and building on each others' efforts, whole complex machines will be designed on the timescale of mere millennia - no, centuries!

Skeptic:  What?

Believer:  You heard me.

Brain Emulation and Hard Takeoff

The construction of a working brain emulation would require, aside from brain scanning equipment and computer hardware to test and run emulations on, highly intelligent and skilled scientists and engineers to develop and improve the emulation software. How many such researchers? A billion dollar project might employ thousands, of widely varying quality and expertise, who would acquire additional expertise over the course of a successful project that results in a working prototype. Now, as Robin says:

They would try multitudes of ways to cut corners on the emulation implementation, checking to see that their bot stayed sane.  I expect several orders of magnitude of efficiency gains to be found easily at first, but that such gains would quickly get hard to find.  While a few key insights would allow large gains, most gains would come from many small improvements.   

Some project would start selling bots when their bot cost fell substantially below the (speedup-adjusted) wages of a profession with humans available to scan.  Even if this risked more leaks, the vast revenue would likely be irresistible.   

To make further improvements they would need skilled workers up-to-speed on relevant fields and the specific workings of the project's design. But the project above can now run an emulation at a cost substantially less than the wages it can bring in. In other words, it is now cheaper for the project to run an instance of one of its brain emulation engineers than it is to hire outside staff or collaborate with competitors. This is especially so because an emulation can be run at high speeds to catch up on areas it does not know well, faster than humans could be hired and brought up to speed, and then duplicated many times. The limiting resource for further advances is no longer the supply of expert humans, but simply computing hardware on which to run emulations.

Emulations Go Foom

Let me consider the AI-foom issue by painting a (looong) picture of the AI scenario I understand best, whole brain emulations, which I'll call "bots."  Here goes.

When investors anticipate that a bot may be feasible soon, they will estimate their chances of creating bots of different levels of quality and cost, as a function of the date, funding, and strategy of their project.  A bot more expensive than any (speedup-adjusted) human wage is of little direct value, but exclusive rights to make a bot costing below most human wages would be worth many trillions of dollars.   

It may well be socially cost-effective to start a bot-building project with a 1% chance of success when its cost falls to the trillion dollar level.  But not only would successful investors probably only gain a small fraction of this net social value, is unlikely any investor group able to direct a trillion could be convinced the project was feasible - there are just too many smart-looking idiots making crazy claims around.

But when the cost to try a 1% project fell below a billion dollars, dozens of groups would no doubt take a shot.  Even if they expected the first feasible bots to be very expensive, they might hope to bring that cost down quickly.  Even if copycats would likely profit more than they, such an enormous prize would still be very tempting. 

The first priority for a bot project would be to create as much emulation fidelity as affordable, to achieve a functioning emulation, i.e., one you could talk to and so on.  Few investments today are allowed a decade of red ink, and so most bot projects would fail within a decade, their corpses warning others about what not to try.  Eventually, however, a project would succeed in making an emulation that is clearly sane and cooperative.

Life's Story Continues

Followup to:  The First World Takeover

As last we looked at the planet, Life's long search in organism-space had only just gotten started.

When I try to structure my understanding of the unfolding process of Life, it seems to me that, to understand the optimization velocity at any given point, I want to break down that velocity using the following abstractions:

• The searchability of the neighborhood of the current location, and the availability of good/better alternatives in that rough region. Maybe call this the optimization slope.  Are the fruit low-hanging or high-hanging, and how large are the fruit?
• The optimization resources, like the amount of computing power available to a fixed program, or the number of individuals in a population pool.
• The optimization efficiency, a curve that gives the amount of searchpower generated by a given investiture of resources, which is presumably a function of the optimizer's structure at that point in time.

Observing Optimization

Followup to: Optimization and the Singularity

In  "Optimization and the Singularity" I pointed out that history since the first replicator, including human history to date, has mostly been a case of nonrecursive optimization - where you've got one thingy doing the optimizing, and another thingy getting optimized.  When evolution builds a better amoeba, that doesn't change the structure of evolution - the mutate-reproduce-select cycle.

But there are exceptions to this rule, such as the invention of sex, which affected the structure of natural selection itself - transforming it to mutate-recombine-mate-reproduce-select.

I was surprised when Robin, in "Eliezer's Meta-Level Determinism" took that idea and ran with it and said:

...his view does seem to make testable predictions about history.  It suggests the introduction of natural selection and of human culture coincided with the very largest capability growth rate increases.  It suggests that the next largest increases were much smaller and coincided in biology with the introduction of cells and sex, and in humans with the introduction of writing and science.  And it suggests other rate increases were substantially smaller.

It hadn't occurred to me to try to derive that kind of testable prediction.  Why?  Well, partially because I'm not an economist.  (Don't get me wrong, it was a virtuous step to try.)  But also because the whole issue looked to me like it was a lot more complicated than that, so it hadn't occurred to me to try to directly extract predictions.

What is this "capability growth rate" of which you speak, Robin? There are old, old controversies in evolutionary biology involved here.