It is likely that they are more complicated, not less. But humans are in the habit of paying attention to squeaky wheels, and then thinking that what they pay attention to is what is most important.

Expand full comment

Personally, I'd say we need to focus more on making the future than predicting it...


Expand full comment

Linden writes:

I am an optimist and a technophile, but I believe that I speak for the vast majority of brain researchers when I express serious doubts about Kurweil’s timetable.

This is confusing. Is he saying that he himself is optimistic, but is obliged to report the contrary consensus of his colleagues (which possibly reduces his confidence in his optimism), or that his colleagues' views are so persuasive that even he, "an optimist and a techophile", agrees with them?

If he is unusually technophilic among brain reseachers, then it isn't unexpected that in speaking for the rest of them (as opposed to himself) he would be more pessimistic; hence the word "but" is inappropriate. (He should have said: "I'm an optimist and a technophile, so it's for the rest of my profession and not myself that I'm speaking when I express doubts about Kurzweil.") On the other hand, if he, as a technophile, agrees with his non-technophile colleagues, then the rhetorical emphasis should be on his own unexpected pessimism, not the expected pessimistic consensus, and he shouldn't need to point out that in expressing pessimism he "speaks for" other (non-technophile) brain researchers (whose consensus can already be taken for granted to be on the pessimistic side).

It would be an appropriate rhetorical formula for a member of the non-technophile consensus, against a technophile: "Linden may agree with Kurzweil -- after all he's an optimist and a technophile -- but I believe I speak for the majority of brain researchers when I say that Kurzweil is wrong." Not for a technophile himself unexpectedly agreeing with the consensus. Such a person should rather say "I'm an optimist and a technophile, but even I agree with the standard pessimism of brain researchers."

Expand full comment

Yes Kurzweil seems too optimistic, but rather than criticizing Kurzweil it seem far more useful for Linden to offer his own best expert estimates.

Huzzah, Prof. Hanson. Reminds me of expert critiques of Dr. Aubrey De Grey, a few years ago.

Expand full comment

Interesting. Of course I was partly kidding. However I have spent my life in the biologic/medical area. There is not a complete account of the way the basic organs or even cells interact.Since they act silently and with little influence from the conscious mind they don't get much respect. It is not clear to me that they are any less sophisticated than the human brain itself which in many ways is the most poorly perfected organ. The second most imperfect organ is the skin.

Expand full comment

Frankly, this is a great idea just for what it would contribute to medical research, never mind upload tech. Something like your crash program, perhaps on a smaller scale, already exists under the name "systems biology".

Expand full comment

Wait, let's upload colons first.What is always so puzzling about these discussions is they are based on unexamined assumptions and mental shortcuts. The one thing these EMs will have is intelligence and awareness.But we have no idea of what awareness consists. No one knows how to enumerate the phenomenon of "intelligence" Is it being good at math or an ability to understand complex metaphors? Computers are great at math but absolutely stink at understanding metaphors.

So we are going to build a computer that is so powerful that it can crush this problem with its sheer computing power. This is the province of computer nerds and sci fi fans. It partakes of the fallacy of infinite extrapolation plus inept comparisons between digital electronics and biologic systems.

Biologically the brain has more in common with the colon than a computer.The colon has a complex nervous system that reacts to biologic stimuli and social stimuli as well as hormonal and chemical influences in an all but unconscious but very organized manner. Every normal person has one whether a genius or a dolt. I Suggest we take things step by step. before indulging wild claims speculation about EMs.

I propose that we begin a crash program so that by the late 2030s, we will be able to routinely scan an individual’s colon with such molecular precision and with such a complete understanding of the rules underlying neuronal function and plasticity that we will be able to “upload” our colons life into a vastly powerful and capacious future computer. By starting with a simple system we can then we can proceed through the other organ systems until we can upload the brains of creative geniuses for the benefit of all humanity.

Expand full comment

> How would we know what inputs to feed the modeled system?

Why do you care? The brain has demonstrated tremendous flexibility. Eyes using your tongue, spatial perception using electricity on your head, geographic orienting and navigation using a buzzing device, echolocation by clicking, goggles that invert your vision and eyes that adapt to that in hardly any time... Just this writing and your reading demonstrates how flexible the symbol-processing of the brain can be with practice!

Expand full comment

They caught him stealing paperclips. Rather, his em was maximizing them through theft.

Expand full comment

But all we need for brain “uploads” (= emulations) are good enough models of individual cell input/output/state relations

So you say. Others* argue we'll need to understand what the parts actually do in more detail. This would obviously make non-upload AGI more likely.

*Namely, everyone else I've seen discussing the issue, though that does seem like a small sample size.

Expand full comment

It seems we could very easily end up with either “brain-dead,” “insane,” or “perpetually tortured” EMs unless we have a pretty comprehensive and detailed picture of how the non-neurological world hooks up with the neurological world.

I find it hard to believe that an EM's pain or insanity would be something we could not determine by fairly casual inspection and thus halt the simulation within a few of the EM's subjective seconds. Only deliberate cruelty or complete apathy would result in the kind of scenario you are predicting. Pain is a very strong signal that commandeers the attention of the conscious mind and produces recognizable reactions.

Note also that EMs can be (and in the beginning will be, due to hardware limitations) run much slower than biological entities, so one could study a second of their time for days or years on end. This is part of what makes them so attractive for studying -- slow motion and reversibility, combined with the ability to access all of the data without scanning, makes it much more likely that we can reverse-engineer the essential algorithms and thereby avoid the ones that cause instability or distress.

Expand full comment

You can get a snapshot of a personality, but all you upload is that snapshot. You don’t get the person in question, all you get is a scan of that person, and thus a copy.

This so-called "snapshot" would change afterwards based on the results of the simulation. The primary difference is you could create a backup of the digitized personality at any given point in time, without having to go through all that messy business of scanning them.

Then you have what happens to that scan in the limited venue of a computer. For a personality is not only the product for a memory, but of the interaction of a human mind with an external world.

Most of my imagination has no direct interface to the physical world. I have to use crude implements like my hands, voice, eyes, etc. to manipulate and perceive reality. Replicating (and exceeding) that degree of interactivity would fall quite short of miracle territory.

Would it be me? Debatable. Would it be a person? Not so much.

Expand full comment

your armchair science misses many details that require more than a miracle.

Such as?

Expand full comment

It's generous of you to defer to an brain science expert when you so clearly understand what "we need for brain 'uploads.'” On the other hand the future may be shiny and your armchair science misses many details that require more than a miracle.

Expand full comment

Clearly you need a prediction market.

Honestly though, Kurzweil is way too optimistic. I'd say nuclear war is more likely than this, although I wouldn't say any serious predictions unless I were a neuroscientist. People from other professions tend to overestimate practicality of any "cool" technology. So many people have predicted the revolution of speech recognition but the practical problems keep piling up. Kurzweil has been too optimistic on most of his predictions. And he doesn't take into account the inertia created by government inefficiencies, direct and undirect.

If a neuroscientist is stating serious doubts about these predictions, I'd say the reality is even more pessimistic, given that usually technology's researches are the ones that are too optimistic. Usually there tends to be a myriad of practical problems

And I think the reason people are not so interested in this is the same reason people are not really interested in investing or betting on the issue. There're too many practical issues.

p.s. I think from ethical point of view your discussion with Karl Smith was interesting though.

Expand full comment