Comments on: What Core Argument?

By: Overcoming Bias : Distrusting Drama

Overcoming Bias : Distrusting Drama — Mon, 05 Dec 2011 13:22:07 +0000

[...] fully accept Eliezer’s AI foom estimates; I’ve explained my reasoning most recently here, here, and here. But since we both post on disagreement meta-issues, I should discuss some of my [...]

By: Tim Tyler

Tim Tyler — Thu, 11 Dec 2008 20:03:01 +0000

Why would developers so severely misjudge their control of [superintelligent growth]

Most of today’s developers don’t worry much because they don’t have to – the chance of any one of them creating a superintelligence soon is miniscule. The job of speculating on what might happen 20 years or so in the future is one for philosophers, not coders.

By: Eliezer Yudkowsky

Eliezer Yudkowsky — Thu, 11 Dec 2008 17:35:11 +0000

Surely, impact is measured not in how many partisans you started with, but in opinion shifts.

By: Tiiba

Tiiba — Thu, 11 Dec 2008 17:01:09 +0000

Now, I’m not saying that Robin is stupid. It’s just that Eliezer is so amazing.

By: James Andrix

James Andrix — Thu, 11 Dec 2008 16:33:19 +0000

Zubon:
I think we’re on the same page. I meant ‘that large’ in comparison to our advantage in other design spaces. We went from kites and balloons to spaceplanes in under 100 years, and we’re even better than that at microprocessor design. We could take this to mean that intelligence is very good at these things, but maybe not so good at improving machine vision algorithms (but still much better than evolution).

By: Thomas Nowa

Thomas Nowa — Thu, 11 Dec 2008 16:01:48 +0000

@Jeff,and @Tiiba: I couldn’t disagree with you either of you more.

Why would you value debate where both sides are in agreement? Only through disagreement and discussion can true debate take place. The contrasting views of Robin and Eliezer are what make this blog thought-provoking and worth reading.

By: advancedaltruist

advancedaltruist — Thu, 11 Dec 2008 15:43:29 +0000

I’m gonna have to go ahead and disagree with you there. I only come here for Eliezer’s posts. Everybody else, even Robin, is a toolbag compared to him. It’s almost annoying how consistently he hits the nail on the head.

By: Vladimir Nesov

Vladimir Nesov — Thu, 11 Dec 2008 15:35:26 +0000

More on architecture vs. content and hard takeoff as a specific technical problem. It might be wrong to think about seed AI as a content-producer, that makes putative stuff like economies do, where you can see improvements and technologies as goods. People project abstractions on the world around them, if they do something, they usually optimize for an abstraction that is statically attached to that thing. You make a processor, a thing that satisfies an abstraction of processor. You perform an operation described by an abstraction on other abstractions. This style of development is itself a specific algorithm of rationality, this is what works for us, this is what we are capable of doing. Economic analysis of this process is static analysis of this algorithm, a specific system operating by more or less simple rules.

If AI starts to invent algorithms for its own cognition, and it scales not by copying little black boxes and integrating them in the old algorithm of economy, but by expanding its mind, then you are in trouble. The activity of AI consists not in putative actions that produce stuff, it consists in following whatever cognitive algorithm previous incarnations of that AI came up with. External activity of the AI is as much operation of its mind as its internal activity, and its mind doesn’t work on economy, it works on novel algorithms optimized for each specific context. Performing static analysis of this algorithm isn’t going to yield simple laws, apart maybe from what physics, information theory and computational complexity can say on the topic, and this is orders of orders of magnitude beyond what we saw.

By: Billy Brown

Billy Brown — Thu, 11 Dec 2008 15:17:56 +0000

Interesting that this discussion bogged with so little progress – this doesn’t exactly inspire confidence in the ability of would-be rationalists to resolve complex issues through discussion.

For what it’s worth, I think the key questions of fact here mostly revolve around the issue of what a mind with human-level intelligence would look like. Eliezer is apparently of the opinion that an AGI is a complex system of specialized modules, where some modules (like vision) do complex but well-defined processing with O(N) to O(logN) performance, while others are best viewed as polynomial approximations of various NP-complete search problems. In this view it’s obvious that once the AGI becomes competent to write AGI code it can rapidly scale up to use any available hardware, and a lot of his other claims about the capabilities of such an AGI rest on fairly sort chains of inference.

Robin obviously doesn’t hold this view. From his writings to date I can’t tell if he has a different model in mind, or if he just considers the whole question unanswerable at this point. But this seems to be the most fundamental technical issue in play.

By: Zubon

Zubon — Thu, 11 Dec 2008 15:11:03 +0000

That we don’t have AI already seems to be evidence that Intellignce might not have that large an advantage over evolution in mind design.

Define “that large”? Intelligence has been on the project for something approaching a century. Evolution has had multicellular life for about a billion years on this planet. Perhaps that is what you mean: intelligence may not be 10,000,000 times as quick. Many of us will be disappointed if intelligence turns out to be 1,000,000 times as quick, leaving us to wait most of a millennium.