83 Comments

Software geek for 20+ years. BS in CS with minor in AI. My gut reaction on reading this post, FWIW:

Creating an em strikes me as a Very Hard Problem. My money is on it not happening, period. That said, if it were to happen, the various problems solved along the way to making an em a reality would make the idea of using an em to do software engineering ridiculous. Instead, you'd branch off some of these intermediate innovations to make software-engineering-specific AI applications that wouldn't be very em-like. Asking how you'd use an em to do software engineering is like asking how much hay you'd need to feed a horseless carriage.

FWIW: I'm currently working with a company that deals with cutting-edge Hard Problems involving huge amounts of data distributed all over the place and used by tons of people and managed by an ecosystem of a gazillion feisty bits of difficult software that nobody really understands in the aggregate. A lot of their work these days is going in to trying to make the status of this ecosystem more easily processable by the human neurological system through creative data visualization techniques... in other words, trying to finesse the boundary between human brain data-processing ability and computer big-data.

Expand full comment

I think the most fundamental change to software tools in this scenario will be a focus shift from tactical economic activities to the optimization of experimentation, discovery and simulation methods.

I think this would be the nexus of the battle between EM's. This could even lead to evolutionary differentiation between EM's.Any real advances beyond speed will still be constrained by the experimental environments the EM's can create. Especially as the multivariate complexities of the simulations grow exponential or slower yet if the carbon/hydrogen system can't be modelled accurately, then discovery will still have a analog speed limit.

I think a core focus of EMs might be to either minimize serendipity or quantify the optimal randomness required to create sufficient serendipity to create answers to primary research questions.

Expand full comment

Most likely yes.

There may persist small colonies of flesh-and-blood humans, though. I'd expect that the larger em world would treat them like we treat, say, the Amish today. Ems wouldn't make any effort to kill the meatspace humans off, or even steal their stuff, but also wouldn't take meatspace humans very seriously.

Expand full comment

(begin attempted humorous remark) Maybe all I *really* care about is whether Robin has long term archiving enabled (end attempted humorous remark)

Expand full comment

You're suggesting the EM version of FOOM / the singularity. Robin has already extensively discussed that scenario. E.g. http://wiki.lesswrong.com/w...

(FOOM is far more likely with designed artifacts like AIs, than with black-box copies like EMs.)

Expand full comment

I say, coming from a fairly detailed knowledge of compiler architecture, that compilers for most programming languages are not inherently serial. (Some programming languages are exceptions, but they are exceptions for bad reasons, not good ones.)

Debuggers can be exactly as parallel as the thing that is being debugged, and no more.

Expand full comment

ANN's? Human brains? If human brain don't foolish why would a functional duplicate?

Expand full comment

Others have suggested compilers and debuggers as inherently serial.

Expand full comment

I don't think there are any widely-used tools with inherently serial parts (as opposed to being serial just because parallelizing takes work).

Expand full comment

That's right, thanks for clarifying.

Expand full comment

If you can copy, emulate, and fork brains, wouldn't flesh & blood humans quickly become obsolete and extinct, as they are comparatively slow, unreliable, inefficient, and mortal?

Expand full comment

"what distinguishes the em-world from most of human history is the sharp decline in welfare em personalities experience."

Yes, people can get used to being poor if it has always been that way, but when they remember better times that takes a huge psychological toll. The recent situation in Greece is a prime example, the country is still relatively rich but it has declined and people lost their certanties which has made the population much more pessimistic and depressed than the population of fast-growing and developing Chile, even though Chile is exactly as rich as Greece in absolute terms.

Expand full comment

neural networks become over-specialised and loose adaptability after working on something too long. I believe he is reffering to this effect for ems in b)

Expand full comment

Most products allow the first developer to do first use and improvement. Doesn't usually add up to natural monopoly.

Expand full comment

Maybe a better question is whether one corporate entity would monopolize all ems. It seems to me like there would be an em monopoly, because the natural monopoly that goes to the first inventor could be quickly extended by using the ems to improve the software engineering that creates the em itself.

Expand full comment

They're all valid considerations. You've convinced me that c) is a serious issue, but as I explained in another comment I think it will be mostly easier to squeeze efficiency out of faster tools than larger software teams. Given the magnitude of EM speedups, I think a) is unlikely in all but the most gigantic projects. In an EM world software could be developed almost arbitrarily quickly, given enough resources. If the marginal constraining factor is cost not time that favors sequential.

I'm no where near qualified to comment on b). But my understanding is that near all brain aging is low-level biological: plaque buildup, DNA damage, neurotransmitter depletion. oxidative stress, cell death, etc. I know you largely model EMs as black-boxes simulations. But the above mechanisms seem totally unrelated to replicating the brain's mental processes with fidelity. It seems a simulated brain could run for orders of magnitude longer subjective time than a physical one.

Expand full comment