Discover more from Overcoming Bias
Software systems are divided into parts, and we have two main ways to measure the fraction of a system that each part represents: lines of code, and resources used. Lines (or bits) of code is a rough measure of the amount of understanding that a part embodies, i.e., how hard it is to create, modify, test, and maintain. For example, a system that is more robust or has a wider range of capacities typically has more lines of code. Resources used include processors, memory, and communication between these items. Resources measure how much it costs to use each part of the system. Systems that do very narrow tasks that are still very hard typically take more resources.
Human brains can be seen as software systems composed of many parts. Each brain occupies a spatial volume, and we can measure the fraction of each brain part via the volume it takes up. People sometimes talk about measuring our understanding of the brain in terms of the fraction of brain volume that is occupied by systems we understand. For example, if we understand parts that take up a big fraction of brain volume, some are tempted to say we are a big fraction of the way toward understanding the brain.
However, using the software analogy, brain volume seems usually to correspond more closely to resources used than to lines of code. For example, brain volumes seem to have roughly similar levels of activity, which isn’t what we’d expect if they corresponded more to lines of code than to resources used.
Consider two ways that we might shrink a software system: we might cut 1% of the lines of code, or 1% of the resources used. If we cut 1% of the resources used via cutting the lines of code that use the fewest resources, we will likely severely limited the range of abilities of a broadly capable system. On the other hand, if we cut the most modular 1% of the lines of code, that system’s effectiveness and range of abilities will probably not fall by remotely as much.
So there can be a huge variation in the effective lines of code corresponding to each brain region, and the easiest parts to understand are probably those with the fewest lines of code. So understanding the quarter of brain volume that is easiest to understand might correspond to understanding only 1% or less of lines of code. And continuing along that path we might understand 99% of brain volume and still be a very long way from being able to create a system that is as productive or useful as a full human brain.
This is why I’m not very optimistic about creating human level AI before brain emulations. Yes, when we have nearly the ability to emulate a whole brain, we will have better data and simulations to help us understand brain parts. But the more brain parts there are to understand, the harder it will be to understand them all before brain emulation is feasible.
Those who expect AI-before-emulations tend to think that there just aren’t that many brain parts, i.e., that the brain doesn’t really embody very many lines of code. Even though the range of capacities of a human brain, even a baby brain, seems large compared to most known software systems, these people think that this analogy is misleading. They guess that in fact there is a concise powerful theory of intelligence that will allow huge performance gains once we understand it. In contrast, I see the analogy to familiar software as more relevant; the vast capacity of human brains suggests they embody the equivalent of a great many lines of code. Content matters more than architecture.
How Good 99% Brains?
There is a lot of overhead involved in training an machine learning (ML) system to understand a problem and solve it well. For most problems it's not efficient because our brains are already like ML systems that know how to solve the given problem, and the info for how to solve the problem is easily expressible.
To take a simple example: imagine writing a program to calculate the area of geometric shapes. You could either train an ML system to understand the general concepts of shape and area, or you could just take a couple minutes to enter in some formulas. The reason we don't try to use ML to solve all of our design problems now is similar to the reason we would directly code this program.
Our brain is a kind of cache, representing some fraction of the intelligence embedded in all the data encountered by our ancestors, and all the data we've seen in our lives. Similarly, any new ML system will be a sort of cached intelligence. We will eventually see AIs doing all design work (either ems, or machine learned ones), but that doesn't mean we'll learn a new system for every problem. Just like animals don't grow a new brain to solve every problem. Doing that would be super inefficient and difficult, compared to the "use the existing cache" solution.
I agree our society is more powerful than evolution. If the competition is between our society hand-coding an AI vs. evolution creating another species at least as smart as humans, then it's no contest, I'll bet on our hand-coding AI.
Deep learning is not just a virtual clone of how evolution works. We can take advantage of our 'industrial' abilities to create better processes for turning data into intelligence. We can also run ML algorithms far faster than evolution.
The situation isn't our society vs. evolution, but our society using hand-coding techniques vs. our society using 'feed lots of data to an intelligently selected learning algorithm' techniques.
ML has basically taken over AI and demonstrated better results than hand-coding on a wide range of AI-related problems. If we want to look at history to extrapolate what techniques will lead to general AI, looking at these problems is a lot more relevant than looking at much simpler industrial problems. This is especially true given that we've only had enough computing power and knowledge to do ML somewhat well for the past ~10 years.
It isn't worth it to use this approach for every type of problem because there is lots of overhead involved. In some cases it will be worth it, in many cases it won't be.
Most computer systems people build are very simple in comparison to brains. If you're writing a program to calculate the area of a geometrical shape, it's overkill to try to train a neural net to understand the general concepts of shape and area, since you already understand them (thanks to evolution and to all the learning your brain has absorbed since you came into existence) and the knowledge is already represented very compactly in a way that you can transfer to a computer in a few minutes.
Think of a brain or an AI system as a sort of cache that represents some fraction of the intelligence embedded in lots of raw data. Once this 'intelligence cache' exists, using it for problems similar to those it's good at solving can be way more efficient than creating a new cache. When you start encountering sufficiently hard or different problems, a new cache may be warranted.
I think AI will eventually take over all design work (either ems, as Robin writes about, or machine learned systems as I think is more likely), but we won't create a new ML systems for each design problem just as animals don't generate new brains for each problem they face.
I agree, our society is way more powerful than evolution. I am not saying evolution is the most efficient way of creating intelligent systems. We can use some principles from evolution and combine them with our own techniques to create better systems. Deep learning isn't just virtual evolution. What they share is the creation of a very complex system by starting with a simple learning architecture, feeding it lots of raw data, and allowing it to adapt to capture the intelligence embedded in this data.
Maybe there's a smarter "industrial" way to create general AI, but I don't see it, and I think the path I described in my original post will arrive sooner than ems.