Of course, unfortunately for Eliezer, this also means that there is very little evidence regarding his proposed timeframe: Roger Schank and Daniel Dennett could easily turn out to be right.

How do old-timers address Kurzweil’s argument about how exponential growth in computing power will make AGI feasible for the first time in the mid-21st century?

If we had a computer today that had infinite memory, and could give the results to any terminating computation in zero time, we would not know how to build an AGI with it. (Some people are of the opinion that some type of lookup-table or theorem-prover could succeed in this case, but I disagree. There is not enough data for a lookup table, and we wouldn’t know how to formalize the world for the theorem prover.)

To clarify (not that anybody cares), when I wrote “defining what ‘A’ and ‘B’ are in, P(A|B)” what I mean is that I want to see how this way of looking at reasoning doesn’t fail for the same reasons Eliezer (accurately IMO) refers to GOFAI as “suggestively named lisp tokens”. Bayesian update may be more sophisticated than pure deduction but the reference issue is what I’m really keen on understanding.

I agree that there was extreme overoptimism in being able to understand cognition in the early days of AI, but they quickly realized things were not as simple as they seemed when they failed so miserably. And even in the early days, nobody would have accepted the much stronger belief you stated that “introspection is infallible,” which was my point.

I think I agree with you your sentiment, but expressing that as “introspection is infallible” is profoundly misleading.

The infallibility of introspection was a central belief in philosophy for hundreds of years. Most people nowadays don’t explicitly endorse the belief but their beliefs about how we should approach and understand the mind are clearly shaped by people who did.

Notice that the vision I sketched goes too far. Provers, such as ACL2, can show that algorithms compute the same function, but they do not prove results about space and time requirements. They cannot express the idea than one algorithm is faster, or that another uses less memory. (Well, actually they can, you code an instrumented interpreter for the algorithms and prove results about the interpreter, but that is my point, there is another level required.) So my vision is not the next step, but since it builds on stuff that ACL2 cannot do, it is two steps on from current research. Also my vision is well short of general AI.

My opinion is that we are conducting AI research three or more conceptual levels below where the action is, and can therefore make no direct progress. We can only enlarge and depend the computing culture, with the hope of moving up a level at some time in the future. Meanwhile you can download ACL2 from the University of Texas and get a feel for the state of the art. Then you can have an opinion too!

I sent Pinker an e-mail saying that it sounded like he was looking for a general purpose ‘Upper Ontology’:

Upper Ontology

Pinker’s comment:

“Yes, I agree that what I am calling a language of thought is closely related to what computer scientists call an ontology.”

I dropped the concept of a putative ‘Universal Parser’. Pinker’s comment:

“I’m not sure whether a universal parser is feasible (in practice – in principle, I’d insist that it is). As I note in chapter 8 (and in the “Talking Heads” chapter in The Language Instinct), sentence interpretation in context requires considerable knowledge about the speaker’s intentions, which may require duplicating a good part of the speaker’s social and cultural knowledge base. That doesn’t seem to be that easy to implement, especially if it is meant to apply cross-culturally, to any language. But perhaps some day.”

Upper Ontology. Universal Parser. Hmm. Sounds a possible new ‘big insight’ into AGI.

An Upper Ontology for General Purpose Reality Modelling.

Hee hee…

No, but look at Microsoft or Intel.

Of course the first seed AI being the ancestor of the last AI is far from a certain outcome.

E.g. maybe the builders of the the first seed AI will cripple it with takeoff constraints – and so inadvertently allow a subsequent AI to take over before its air supply can get cut off, and its lunch can be eaten.

Also, the rise to power of these things may take more than “a few months”.