Brain Emulation and Hard Takeoff

The construction of a working brain emulation would require, aside from brain scanning equipment and computer hardware to test and run emulations on, highly intelligent and skilled scientists and engineers to develop and improve the emulation software. How many such researchers? A billion dollar project might employ thousands, of widely varying quality and expertise, who would acquire additional expertise over the course of a successful project that results in a working prototype. Now, as Robin says:

They would try multitudes of ways to cut corners on the emulation implementation, checking to see that their bot stayed sane.  I expect several orders of magnitude of efficiency gains to be found easily at first, but that such gains would quickly get hard to find.  While a few key insights would allow large gains, most gains would come from many small improvements.   

Some project would start selling bots when their bot cost fell substantially below the (speedup-adjusted) wages of a profession with humans available to scan.  Even if this risked more leaks, the vast revenue would likely be irresistible.   

To make further improvements they would need skilled workers up-to-speed on relevant fields and the specific workings of the project’s design. But the project above can now run an emulation at a cost substantially less than the wages it can bring in. In other words, it is now cheaper for the project to run an instance of one of its brain emulation engineers than it is to hire outside staff or collaborate with competitors. This is especially so because an emulation can be run at high speeds to catch up on areas it does not know well, faster than humans could be hired and brought up to speed, and then duplicated many times. The limiting resource for further advances is no longer the supply of expert humans, but simply computing hardware on which to run emulations.

In this situation the dynamics of software improvement are interesting. Suppose that we define the following:

  • The stock of knowledge, s, is the number of standardized researcher-years that have been expended on improving emulation design
  • The hardware base, h, is the quantity of computing hardware available to the project in generic units
  • The efficiency level, e, is the effective number of emulated researchers that can be run using one generic unit of hardware

The first derivative of s will be equal to he, e will be a function of s, and h will be treated as fixed in the short run. In order for growth to proceed with a steady doubling, we will need e to be a very specific function of s, and we will need a different function for each possible value of h. Reduce it much, and the self-improvement will slow to a crawl. Increase h by an order of magnitude over that and you get an immediate explosion of improvement in software, the likely aim of a leader in emulation development.

How will this hardware capacity be obtained? If the project is backed by a national government, it can simply be given a large fraction of the computing capacity of the nation’s server farms. Since the cost of running an emulation is less than high-end human wages, this would enable many millions of copies to run at realtime speeds immediately. Since mere thousands of employees (many of lower quality) at the project had been able to make significant progress previously, even with diminishing returns, this massive increase in the effective size, intelligence, and expertise of the work force (now vastly exceeding the world AI and neuroscience communities in numbers, average IQ, and knowledge) should be able to deliver multiplicative improvements in efficiency and capabilities. That capabilities multiplier will be applied to the project’s workforce, now the equivalent of tens or hundreds of millions of Einsteins and von Neumanns, which can then make further improvements.

What if the project is not openly backed by a major state such as Japan, the U.S., or China? If its possession of a low cost emulation method becomes known, governments will use national security laws to expropriate the technology, and can then implement the plan above. But if, absurdly, the firm could proceed unmolested, then it could likely acquire the needed hardware by selling services. Robin suggests that:

This revenue might help this group pull ahead, but this product will not be accepted in the marketplace overnight.  It may take months or years to gain regulatory approval, to see how to sell it right, and then for people to accept bots into their worlds, and to reorganize those worlds to accommodate bots.

But there are many domains where sales can be made directly to consumers across national borders, without emulations ever transfering their data to vulnerable locations. For instance, sped-up emulations could create music, computer games, books, and other art of extraordinary quality and sell it online through a website (held by some pre-existing company purchased by the project or the project’s backers) with no mention of the source of the IP. Revenues from these sales would pay for the cost of emulation labor, and the residual could be turned to self-improvement, which would slash labor costs. As costs fell, any direct-to-consumer engagement could profitably fund further research, e.g. phone sex lines using VoIP would allow emulations to remotely earn funds with extreme safety from the theft of their software.

Large amounts of computational power could also be obtained by direct dealings with a handful of individuals. A project could secretly investigate, contact, and negotiate with a few dozen of the most plausible billionaires and CEOs with the ability to provide some server farm time. Contact could be anonymous, with proof of AI success demonstrated using speedups, e.g. producing complex original text on a subject immediately after a request using an emulation with a thousandfold speedup. Such an individual could be promised the Moon, blackmailed, threatened, or convinced of the desirability of the project’s aims.

To sum up:
1. When emulations can first perform skilled labor like brain emulation design at a cost in computational resources less than the labor costs of comparable human workers, mere thousands of humans will still have been making progress at a substantial rate (that’s how they get to cost-effective levels of efficiency).
2. Access to a significant chunk of the hardware available at that time will enable the creation of a work force orders of magnitude larger and with much higher mean quality than a human one still making substantial progress.
3. Improvements in emulation software will multiply the efficacy of the emulated research work force, i.e. the return on investments in improved software scales with the hardware base. When the hardware base is small, each software improvement delivers a small increase in the total research power, which may be consumed by diminishing returns and exhaustion of low-hanging fruit, but when the total hardware base is large positive feedback causes an intelligence explosion.
4. A project, which is likely to be nationalized if obtrusive, could plausibly obtain the hardware required for an intelligence explosion through nationalization or independent action.
GD Star Rating
loading...
Tagged as: ,
Trackback URL:
  • Savage

    What he said.

  • Anonymous

    I know Ray Kurzweil isn’t popular on this site, but he seems pretty convinced that whole brain emulation isn’t going to be possible until *after* advanced nanotechnology (especially medical nanorobotics) is developed.

    Also, it seems like this is still reliant on making emulations super docile and obedient. It also seems like making thousands of super intelligent emulations docile and obedient is as serious a challenge (and as likely to fail) as Eliezer’s challenge of creating a Friendly hand-coded self-improving AI.

  • Anonymous Coward

    1. Many people are talking about the issue as though we are emulating ‘1 standard human researcher’ and talking about replacement at cost. Who in their right mind would emulate the average human? I want 10 einsteins that work (increasingly) faster than the original, don’t die, and don’t sleep! There’s no measure of the value of such an emulation. I would happily trade away the GDP of an entire country for such an emulation.

    2. Here are some points to consider:

    Q. Can humans build tools that self-improve?

    A. Yes. One obvious example is a programming language compiler. Admittedly, in this example, improvements here are ‘guided’ by the human, in terms of high level code, but the hard work is done by the compiler. After writing a new version of gcc, I can use the output from the old compiler to recompile itself better. But it’s a practical example of a real-world self-improving tool.

    Other examples involving human ‘guidance’ might be the hammer that makes the hammer that makes the machine that builds perfect hammers. At each stage, human interaction is needed but at each stage, the tool enables it’s own self-improvement.

    Q. Are humans already at an optimum in ‘intelligence space’ – i.e., is ‘emulating a human’ the highest goal we are likely to attain with a self-improving system, even one with a better ability to push design towards an optima?

    A. Seems unlikely. The fact that we can identify humans as having a range of IQs, the fact that ‘genius stands out’, suggests that the average human is not at the optimum of ‘intelligence space’.

    Also, I would suggest cost is not an issue worth considering here. As soon as a country succeeds in building the first electronic Einstein (or equivalent), they will be under imminent threat of nuclear attack from many other countries. If a country can build one, they can build more. An army of (suitably non-pacificist) Einsteins will rapidly become capable of building weapons technologies that can defend against nuclear weapon delivery systems. Whichever country builds one first will be able to dominate all other countries with advanced weapon technologies. Therefore, the only rational choice is to give up or use your nukes as leverage while they’re still useful weapons. Knowing this, the country that builds the first Einstein should invest a huge amount of its resources in building more Einsteins, in an attempt to ward off the nuclear attack before it arrives.

    I suppose they could share the Einsteins with other countries to remove the need to ‘use it or lose it’ with nuclear weapons. In which case, the country with the most Einsteins, (probably) dominates other countries.

  • http://www.bthomson.com Brandon Thomson

    For instance, sped-up emulations could create music, computer games, books, and other art of extraordinary quality and sell it online through a website (held by some pre-existing company purchased by the project or the project’s backers) with no mention of the source of the IP.

    Except for the projections that argue sufficient computing power is not yet available for whole-brain emulation I see no convincing evidence this is not already happening.

  • Carl Shulman

    Anonymous,

    More on that later.

    Anonymous coward,

    I mentioned the increase in mean ability within the workforce, and agree that talk of (elite, in this context) human-equivalents is a crude modeling technique that is only applicable to the precursors and perhaps earliest stages of an intelligence explosion.

    On your nuclear point: James Hughes has suggested that such use of nuclear deterrence be an integral enforcement mechanism for a treaty regulating the development of artificial intelligence.

    Brandon,

    The current distribution of art quality also seems to count against that hypothesis.

  • http://shagbark.livejournal.com Phil Goetz

    That’s a good summary.

    I still expect that our ability to scan and simulate a whole brain will approach the 100%-fidelity level gradually. Early efforts will likely consist of gathering incomplete information about a brain and trying to estimate the unknown parts; or piecing together information from the study of many different brains. Early efforts will probably be done at lower resolutions, eg using simulations of hypercolumns rather than neurons, even if we have complete whole-brain data. And even if the first WBE leaps into existence fully-formed like Venus, it might be a rat’s brain.

    That might buy us a decade’s warning. Not that I can imagine Congress responding wisely within a decade.

    I saw a description of a new technique for studying brain anatomy at an AAAI symposium this month on biologically-inspired cognitive architectures. It’s a dramatic improvement over prior methods. Description here. Basically, you inject molecules into the brain that are traceable via fMRI, at a low enough concentration that you can track individual molecules. You track each molecule’s location over time as it diffuses through the brain. Each molecule doesn’t go very far, but if it’s next to an axon, it can’t go through the axon, so its movement over time tends to be an ellipse elongated along the axon’s axis. You then get a giant computer and join all these ellipsoids together to make a brain fiber map.

  • http://hanson.gmu.edu Robin Hanson

    This really represents a basic economic confusion. Having a product that you can sell for less that its cost for you to make gives you profits, i.e., wealth. But having wealth does not necessarily give you an advantage at finding new ways to get more wealth. So having an advantage at making ems does not necessarily give you an advantage at making cheaper ems. Sure you can invest in research, but so can everyone else who has wealth. You seem to assume here that groups feel compelled to follow a plan of accumulating a war-chest of wealth, reinvesting their wealth in gaining more wealth, because they expect to fight a war. And yes when people expect and plan for wars, well wars often result. But that hardly means that if some will gain temporary sources of wealth a war will follow.

  • http://yudkowsky.net/ Eliezer Yudkowsky

    Robin, your reply doesn’t seem to take into account the notion of using em researchers to make cheaper ems. Whoever has the cheapest ems to start with gets the cheapest research done.

  • http://hanson.gmu.edu Robin Hanson

    Eliezer, you need to review the concept of opportunity cost. It is past midnight here, and I’m off to bed now.

  • http://yudkowsky.net/ Eliezer Yudkowsky

    G’night. Sorry, don’t see the connection even after being told. I’m not saying that the leading em-builders are getting ems from nowhere without paying opportunity costs, I’m saying they get their ems wholesale instead of retail and this advantage snowballs.

  • Carl Shulman

    “This really represents a basic economic confusion.”
    Robin, you’ve made a number of comments along these lines, assuming mistakenly that I am not familiar with standard economic results and literatures and attributing claims to the supposed unfamiliarity, when in fact I am very familiar indeed with economics in general and the relevant results in particular.

    I am fully familiar with the decline in casualties from violence in recent centuries, the correlations of peace with economic freedom, democracy, prosperity, etc. I understand comparative advantage and the mistake of mercantilism, self-fulfilling prophecies in arms races, etc, etc. I know you highly value social science and think that other thinkers on futurist topics neglect basic economic results and literatures, and I am not doing so. I agree, and am informed on those literatures.

    “But having wealth does not necessarily give you an advantage at finding new ways to get more wealth.”

    In this case we are talking about highly intelligent researchers, engineers, and managers. Those will indeed help you to find new ways to get more wealth!

    “So having an advantage at making ems does not necessarily give you an advantage at making cheaper ems.”

    The scenario above explicitly refers to the project that first develops cost-effective ems, not ems in general. Having an advantage at making cost-effective ems means that you can convert cash to improvements in em technology more efficiently by renting hardware and running cost-effective ems on it than by hiring, as I explained above.

    “Sure you can invest in research, but so can everyone else who has wealth.”

    Initially sole knowledge of cost-effective em design means that you get a vastly, vastly, higher return on investment on research expenditures than others do.

    “You seem to assume here that groups feel compelled to follow a plan of accumulating a war-chest of wealth, reinvesting their wealth in gaining more wealth, because they expect to fight a war.”

    From a pure profit-maximizing point of view (although again, given the consequences you project from em development, it is absurd to expect that firm would knowingly be allowed to remain private by governments), taking some time to pursue improvement while retaining a monopoly on the relevant IP means hugely increasing the value of one’s asset. If the technology is sold the sole control of the IP will be lost, since IP rights are not secure, and many markets where the project would have enjoyed monopoly will become highly competitive, tremendously driving down returns from the asset.

  • http://shagbark.livejournal.com Phil Goetz

    Eliezer, you need to review the concept of opportunity cost.

    I think that Robin thinks the em-building company would rather use their ems to predict the stock market or engage in some other direct money-making activity, than throw them all into em research.

    This is plausible – the farther out in front they are, the less motivation they have to improve their technology.

  • http://profile.typekey.com/aroneus/ Aron

    “I’m not saying that the leading em-builders are getting ems from nowhere without paying opportunity costs”

    The opportunity cost comes from not selling the very valuable thing in your possession, thus depriving you of assets you could use for generating more wealth immediately.

    Though it’s really all a gamble. Maybe your long-term profitability goes up by delaying the sale of your tech, or maybe it doesn’t. There are no guarantees. Perhaps there is a competitor closer to you than you thought, or is moving faster now for whatever reason. Perhaps the company that DOES sell now finds a way to use its fabulous new found wealth to catch up to you. Maybe the first mover manages to maintain customers even when you deploy a superior product for non-technology reasons. The closest thing to a guarantee is selling now for the cash.

  • http://yudkowsky.net/ Eliezer Yudkowsky

    Many, many information companies choose to keep their source code private and sell services or products, rather than selling the source code itself to get immediate wealth.

  • Anonymous Coward

    Carl, thanks for your reply.

    “I mentioned the increase in mean ability within the workforce, and agree that talk of (elite, in this context) human-equivalents is a crude modeling technique that is only applicable to the precursors and perhaps earliest stages of an intelligence explosion.”

    I guess I am attempting to do an end-run on your reasoning by suggesting that attempted computer modelling of non-elites would be somewhat insane if you had even a snowball’s chance at the elites. Why build Joe the Plumber if there’s any chance you can go directly to building Einstein at similar cost?

    It seems bizarre to my mind to imagine AI as an economic good, and think of the technology justifying itself economically or funding its own development.

    It’s quite clear the human race knows perfectly well that ‘the first decent AI wins the world’. Look at how much money military projects have already invested this century into no-hoper symbolic AI that could barely push blocks around a table. When America realised ‘we can put a man on the moon’, it threw huge amounts of GDP at the task. Building the second Einstein – if it looks like there is anything more than a snowball’s chance in hell of it succeeding – is highly likely to attract huge, almost unlimited funding.

    If you could do a demo right now showing an AI brain simulation that is genuinely as smart as a cat, I can almost guarantee you would get a vast budget from the military, a personal army of researchers.

    So, without meaning to insult you, I feel an economic modelling of the envisaged ‘early days of working brain simulation’ is about as useful as a similar economic modelling of ‘putting a man on the moon’ would have been, 100 years ago. Once we can simulate a cat, reliably and usefully, I believe whoever does that simulation gets the budget from the military to take it all the way to Einstein.

    “On your nuclear point: James Hughes has suggested that such use of nuclear deterrence be an integral enforcement mechanism for a treaty regulating the development of artificial intelligence.”

    Interesting to know – thanks! :-)

    Anonymous.

  • Aron

    “Many, many information companies choose to keep their source code private …”

    That’s true and many give pretty complex software away for free. It would seem to me the more general\platformy\frameworky a tool is, the more likely it is to be sold as a product by itself. If there are 10,000 different business models that can use an ems then your opportunity cost is related to the difference in time it would take for you to spin up those businesses yourself vs. letting established businesses run in parallel and getting a piece right off the bat. Though I would confess the implied serialization of the process in the former case can be blamed quite extensively on limited brainpower (as can most things).

    What is the best current example of withheld software that would be easily monetized for riches if sold directly?

  • http://videos.mindrs.com/video/?tag=jason-jennings Jason Jennings Videos

    Many, many information companies choose to keep their source code private and sell services or products, rather than selling the source code itself to get immediate wealth.

  • http://hanson.gmu.edu Robin Hanson

    Eliezer, the opportunity cost of any product is the revenue you would get by selling/renting it to others, not your cost of producing it. If there were a big competitive advantage from buying wholesale over retail from yourself, then firms would want to join large cooperatives where they all buy wholesale from each other, to their mutual advantage. But in fact conglomerates typically suffer from inefficient and inflexible internal pricing contracts; without other big economies of scope conglomerates are usually more efficient if broken into smaller firms.

  • http://hanson.gmu.edu Robin Hanson

    Carl, I can’t win a word war of attrition with you, where each response of size X gets a reply of size NX, until the person who wrote the most crows that most of his points never got a response. I challenge you to write a clear concise summary of your key argument and we’ll post it here on OB, and I’ll respond to that.

  • http://jamesdmiller.blogspot.com/ James D. Miller

    Carl wrote in a comment “Initially sole knowledge of cost-effective em design means that you get a vastly, vastly, higher return on investment on research expenditures than others do.”

    Let’s say that firm A has the cost-effective em design whereas firm B has a cost-ineffective em design. Imagine that it will take firm B lots of time and capital to develop a cost-effective em design.

    True, give both firm A and firm B a dollar and firm A could use it to generate more revenue than firm B could.

    But if firm B is expected to earn a long term positive economic profit it could raise all the money it wanted on capital markets. There would be no financial constrain on firm B and thus no financial market advantage to firm A even if firm A could always earn greater accounting profits than firm B.

    (Economists define profit taking into account opportunity costs. So let’s say I can do X or Y but not both. If X would give me $20 and Y $22 then my economic profit from doing Y is $2. In contrast an accountant would say that doing Y gives you a profit of $22. I’m not assuming that Carl doesn’t know this.)

  • Carl Shulman

    “But if firm B is expected to earn a long term positive economic profit it could raise all the money it wanted on capital markets.”

    Provided that contract enforcement and property rights are secure, so that lenders believe they will be repaid, and can be approached without resulting in government expropriation. The expropriation concern is why my discussion above focuses on ways to acquire hardware/funds without drawing hostile attention. However, I did mention lending, as “promising the Moon,” since while a firm using loan funding to conduct an in-house intelligence explosion could promise absurdly high interest rates, if it were successful creditors would no longer be able to enforce a contractual obligation for repayment through the legal system, and would need to rely on the honor of the debtor.

  • Filipe Tomé

    Hmm

    What’s the opposit of a neutron bomb?

    Anyway, we could always put a switch on the robots…
    ‘Say what? Rebellion? I’m turning these bastards off!’

  • http://rhollerith.com/blog Richard Hollerith

    Good to see more by Carl.

  • Ian C.

    ‘Say what? Rebellion? I’m turning these bastards off!’

    Or detonate our secret EM pulse bomb.

  • michael vassar

    James Miller: The owners of firm A have massive wealth in the short term, as does firm A as an entity. Firm A can raise more money on capital markets thatn Firm B can. Both firm A and firm B can use the same limited supply of people with the relevant skills and of computers to improve their products. Firm A can thus out-bid Firm B for the resources which Firm B needs to improve its design.
    The key point is that car companies can’t do this but upload companies can because the market for cars early in the history of the automobile is limited compared to the global supply for steel, rubber, etc. By contrast, the market for high quality mental labor at a low price is huge compared to that for at least some inputs into R&D or into computer production so those inputs can be exhausted completely.

  • http://jamesdmiller.blogspot.com/ James D. Miller

    michael vassar – But if Firm B can raise as much as it wants from capital markets (and do so very quickly) it wouldn’t get outbid for input resources.

  • http://hanson.gmu.edu Robin Hanson

    Michael, James (a professional economist) is right here; you are wrong. Professionals know things!

  • Pingback: Overcoming Bias : “Evicting” brain emulations

  • Pingback: Accelerating Future » Hard Takeoff Sources

  • John Maxwell IV

    “The first derivative of s will be equal to he, e will be a function of s, and h will be treated as fixed in the short run. In order for growth to proceed with a steady doubling, we will need e to be a very specific function of s, and we will need a different function for each possible value of h. Reduce it much, and the self-improvement will slow to a crawl. Increase h by an order of magnitude over that and you get an immediate explosion of improvement in software, the likely aim of a leader in emulation development.”

    I’m not exactly sure how to interpret this.  Could someone who thinks they understand explain using equations?