Discussion about this post

User's avatar
Elliot's avatar

There is a lot of overhead involved in training an machine learning (ML) system to understand a problem and solve it well. For most problems it's not efficient because our brains are already like ML systems that know how to solve the given problem, and the info for how to solve the problem is easily expressible.

To take a simple example: imagine writing a program to calculate the area of geometric shapes. You could either train an ML system to understand the general concepts of shape and area, or you could just take a couple minutes to enter in some formulas. The reason we don't try to use ML to solve all of our design problems now is similar to the reason we would directly code this program.

Our brain is a kind of cache, representing some fraction of the intelligence embedded in all the data encountered by our ancestors, and all the data we've seen in our lives. Similarly, any new ML system will be a sort of cached intelligence. We will eventually see AIs doing all design work (either ems, or machine learned ones), but that doesn't mean we'll learn a new system for every problem. Just like animals don't grow a new brain to solve every problem. Doing that would be super inefficient and difficult, compared to the "use the existing cache" solution.

I agree our society is more powerful than evolution. If the competition is between our society hand-coding an AI vs. evolution creating another species at least as smart as humans, then it's no contest, I'll bet on our hand-coding AI.

Deep learning is not just a virtual clone of how evolution works. We can take advantage of our 'industrial' abilities to create better processes for turning data into intelligence. We can also run ML algorithms far faster than evolution.

The situation isn't our society vs. evolution, but our society using hand-coding techniques vs. our society using 'feed lots of data to an intelligently selected learning algorithm' techniques.

ML has basically taken over AI and demonstrated better results than hand-coding on a wide range of AI-related problems. If we want to look at history to extrapolate what techniques will lead to general AI, looking at these problems is a lot more relevant than looking at much simpler industrial problems. This is especially true given that we've only had enough computing power and knowledge to do ML somewhat well for the past ~10 years.

Expand full comment
Elliot's avatar

It isn't worth it to use this approach for every type of problem because there is lots of overhead involved. In some cases it will be worth it, in many cases it won't be.

Most computer systems people build are very simple in comparison to brains. If you're writing a program to calculate the area of a geometrical shape, it's overkill to try to train a neural net to understand the general concepts of shape and area, since you already understand them (thanks to evolution and to all the learning your brain has absorbed since you came into existence) and the knowledge is already represented very compactly in a way that you can transfer to a computer in a few minutes.

Think of a brain or an AI system as a sort of cache that represents some fraction of the intelligence embedded in lots of raw data. Once this 'intelligence cache' exists, using it for problems similar to those it's good at solving can be way more efficient than creating a new cache. When you start encountering sufficiently hard or different problems, a new cache may be warranted.

I think AI will eventually take over all design work (either ems, as Robin writes about, or machine learned systems as I think is more likely), but we won't create a new ML systems for each design problem just as animals don't generate new brains for each problem they face.

I agree, our society is way more powerful than evolution. I am not saying evolution is the most efficient way of creating intelligent systems. We can use some principles from evolution and combine them with our own techniques to create better systems. Deep learning isn't just virtual evolution. What they share is the creation of a very complex system by starting with a simple learning architecture, feeding it lots of raw data, and allowing it to adapt to capture the intelligence embedded in this data.

Maybe there's a smarter "industrial" way to create general AI, but I don't see it, and I think the path I described in my original post will arrive sooner than ems.

Expand full comment
43 more comments...

No posts