On Wednesday I debated my ex-co-blogger Eliezer Yudkowsky at a private Jane Street Capital event (crude audio here, from 4:45; better video here [as of July 14]).
I “won” in the sense of gaining more audience votes — the vote was 45-40 (him to me) before, and 32-33 after the debate. That makes me two for two, after my similar “win” over Bryan Caplan (42-10 before, 25-20 after). This probably says little about me, however, since contrarians usually “win” such debates.
Our topic was: Compared to the farming and industrial revolutions, intelligence explosion first-movers will quickly control a much larger fraction of their new world. He was pro, I was con. We also debated this subject here on Overcoming Bias from June to December 2008. Let me now try to summarize my current position.
The key issue is: how chunky and powerful are as-yet-undiscovered insights into the architecture of “thinking” in general (vs. on particular topics)? Assume there are many such insights, each requiring that brains be restructured to take advantage. (Ordinary humans couldn’t use them.) Also assume that the field of AI research reaches a key pivotal level of development. And at that point, imagine some AI research team discovers a powerful insight, and builds an AI with an architecture embodying it. Such an AI might then search for more such insights more efficiently than all other the AI research teams who share their results put together.
This new fast AI might then use its advantage to find another powerful insight, restructure itself to take advantage of it, and so on until it was fantastically good at thinking in general. (Or if the first insight were super-powerful, it might jump to this level in one step.) How good? So good that it could greatly out-compete the entire rest of the world at the key task of learning the vast ocean of specific knowledge and insights useful for functioning in the world. So good that even though it started out knowing almost nothing, after a few weeks it knows more than the entire rest of the world put together.
(Note that the advantages of silicon and self-modifiable code over biological brains do not count as relevant chunky architectural insights — they are available to all competing AI teams.)
In the debate, Eliezer gave six reasons to think very powerful brain architectural insights remain undiscovered:
- Human mind abilities have a strong common IQ factor.
- Humans show many specific mental failings in reasoning.
- Humans have completely dominated their chimp siblings.
- Chimps can’t function as “scientists” in human society.
- “Science” was invented, allowing progress in diverse fields.
- AGI researchers focus on architectures, share little content.
My responses: Continue reading "Debating Yudkowsky" »
loading...






