16 Comments

language

Expand full comment

Ahh, yah that does seem like the better interpretation. I was assuming that simply there was a fair bit of overlap in the modules but we were talking about modules in the sense of parts of the brain with specific functional roles but I guess the analogy is with extra proteins not necessary for cell function.

Expand full comment

The argument made in this post is regarding the ability to cut modules without changing behaviour, though. Whether an em needs to jockey for social status in its job seems like a different question from whether specific brain volumes can be deactivated without affecting job performance.

Expand full comment

The efficiency of cutting many modules in ems is the point I was trying to make some time ago about your assumptions about ems behaving largely like normal humans with all the normal human desires, motivations habits etc..

It will be more efficient to cut a substantial number of em's modules and that might affect their behavior in a multitude of ways. For instance productive ems might not have many of the modules people use for jockeying for social status, enjoying watching narratives (TV entertainment etc..) (I take it a lack of sex drive goes without saying). I think this should affect one's analysis of how much we should expect ems to behave like normal humans.

Expand full comment

Then the redundancy would be at the industry level, but it would be there. It wouldn't be the one sleek low-redundancy super-AI improving itself.

Expand full comment

You have a better name?

Expand full comment

I'm not saying we won't need to do the search, I'm saying it can be done at a higher level - more like the level of the whole AI industry rather than within individual AI system instances. I'm claiming that because an industrial economy can consist of much much more specialisation and interdependence than can a biological ecosystem, more of that search can be done 'in advance', with the actual AI systems deployed having less need to access a broad set of capabilities many of which they won't use.

Compare, for example, the fact that cars don't have to be able to reproduce more cars, because we have separate specialised factories to do that. This doesn't mean this manufacturing process doesn't still need to happen - of course it does - but it's done in a very different way than in biological organisms, with much more specialisation and interdependence, and much less redundancy.

Expand full comment

"What can this view say about the future of brains? On ems, it suggests that human brains have a lot of extra capacity. We can probably go far in taking an em that can do a job task and throwing away brain modules not needed for that task."

So an "em" with many modules "thrown away" is still an "em"?

Expand full comment

What are you referring to with "ems"?

Expand full comment

Regarding other artificial intelligences, is there a useful analogue of cultural evolution and what is the current state of the art wrt that?

Expand full comment

I think this is the most accurate description on how "AI" will progress that I've read. It seems terribly unlikely that AI will be discovered in one brilliant algorithm or lucky evolutionary software/machine learning stint. It will likely be messy and incremental, but seem like inexorable progress.

Expand full comment

Insightful. So, to create AI, we need a) A full list of tools - would be interested to have one for human cognitive abilities b) Several different copies of the tool, may de using different approaches for the same task. с) Tool combining subroutine.

Expand full comment

You are in essence assuming we won't need to do a big substantially random search, because we'll have a plan. But how will we learn how to make that plan, if not via search?

Expand full comment

By being able to design artificial intelligence software 'on paper' without having to demonstrate a functional useful product at every stage, I'd expect the AI we build to depend less on this kind of redundancy and be able to be more focused, specialised, and perhaps brittle, and to be much more interdependent rather than having to be self-contained and autarkic. At least, that's how it seems to work when we build artificial replacements for other biological systems.

Expand full comment