Sam Wilson and I did a podcast for his series, on near-far, em econ, and related topics. One topic that came up briefly deserves emphasis: robustness can be very expensive. Imagine I told you to pack a bag for a trip, but I wouldn’t tell you to where. The wider the set of possibilities you needed to handle, the bigger and more expensive your bag would have to be. You might not need a bag at all if you knew your destination was to stay inside one of the hundred largest airports. But you’d need a big bag if you might go anywhere on the surface of the Earth. You’d need a space-suit if you might go anywhere in the solar system, and if you might go anywhere within the Sun, well we have no bag for that.
Pack tools. More generally, this is an 80/20 type problem - for a small cost you can pack enough variety of stuff to cover most of the likely scenarios. Covering the remainder of scenarios is really difficult.
So be reasonably prudent - pack what you can at at a reasonable cost and hope you get lucky.
There are some places where "a bag of money" wouldn't be all that helpful, such as Antarctica, and it also leaves you vulnerable to being robbed. (Another concern I'd have is not knowing the local language.) It's probably okay in most places, though.
I agree. Norway in particular due to its sovereign wealth fund. Seems like a good strategy for those that have the option. It's as close as a state can get to following Joe Teicher's advice to pack a big bag of money.
"A society that understood the importance of prices and complete property rights would be more robust against a slew of risky scenarios, from anti-biotic resistance to peak oil to climate change."
Ehm, I think societies that push a lot of norms (call if indoctrination if you will) are more robust to above-mentioned things. Let's go from the intellectual detail to real world of politics. I'm not american but how many conservatives do you see championing for antibiotic regulation or climate change regulation? Vaccination programs? Obviously certain economic level is needed for this (which requires good enforcement of property rights).
I've long hold the belief that Nordic countries are quite robust to tail risks. The public education (cultural and demographic uniformity too) pushes huge amount of things to your brain. It leads to certain kind of myopic, state-serving thinking but one thing it does achieve is that there is generally less hyperbola here, and less all kinds of crazies. Solving larger coordination problems becomes easier.
I agree that slowing hardware gains could slow the arrival of ems, if hardware costs end up being the limiting factor. I also agree that working ems would help other AI progress. Even so, my guess is that ems come first.
In the podcast you talk about your em scenario. You mention some reason why you expect that non-em strong AI to take a long time (only 5-10% progress to date + probably slowing improvements in hardware and software in the future). But then when you discuss the em scenario, you go straight from why ems should be possible to a situation where they are very cheap. I don't get why the same factors that keep AI progress slow won't also keep em price progress slow. Slowing hardware progress seems like a big problem for ems. Also, it seems plausible to me that the insights needed to optimize em software could accelerate non-em AI progress. Do you think that the ubiquitous em scenario is the most likely scenario for the far future, or do you just think it the most interesting?
Yes sometimes there are win-win choices where you don't have to give up anything to get some things better. So just assume we've done all those obvious wins, and are now considering other choices.
Wealth wouldn't help much with invading aliens or the computer that takes over the world. But yeah, it is a handy approach when dealing with a robust economy.
>Imagine I told you to pack a bag for a trip, but I wouldn’t tell you to where. The wider the set of possibilities you needed to handle, the bigger and more expensive your bag would have to be.
You could take a bag full of money, and just buy what you need when you get there. That isn't completely robust and it isn't as efficient as knowing where you are going and packing what you will need ahead of time, but it still works pretty well. Can that strategy be generalized to dealing with the uncertainty of the future in general? I think so.
It depends on who's providing the robustness. For a single individual or organization, robustness is expensive. But aren't some social systems categorically more robust to future scenarios than others, without clearly being more expensive? A society that understood the importance of prices and complete property rights would be more robust against a slew of risky scenarios, from anti-biotic resistance to peak oil to climate change. And it would cost less than nothing. For example, Germany's flexible labour market reforms made it more robust to aggregate demand shocks. Economic growth made us more robust to weather fluctuations and drought. etc.
Pack tools. More generally, this is an 80/20 type problem - for a small cost you can pack enough variety of stuff to cover most of the likely scenarios. Covering the remainder of scenarios is really difficult.
So be reasonably prudent - pack what you can at at a reasonable cost and hope you get lucky.
There are some places where "a bag of money" wouldn't be all that helpful, such as Antarctica, and it also leaves you vulnerable to being robbed. (Another concern I'd have is not knowing the local language.) It's probably okay in most places, though.
I agree. Norway in particular due to its sovereign wealth fund. Seems like a good strategy for those that have the option. It's as close as a state can get to following Joe Teicher's advice to pack a big bag of money.
"A society that understood the importance of prices and complete property rights would be more robust against a slew of risky scenarios, from anti-biotic resistance to peak oil to climate change."
Ehm, I think societies that push a lot of norms (call if indoctrination if you will) are more robust to above-mentioned things. Let's go from the intellectual detail to real world of politics. I'm not american but how many conservatives do you see championing for antibiotic regulation or climate change regulation? Vaccination programs? Obviously certain economic level is needed for this (which requires good enforcement of property rights).
I've long hold the belief that Nordic countries are quite robust to tail risks. The public education (cultural and demographic uniformity too) pushes huge amount of things to your brain. It leads to certain kind of myopic, state-serving thinking but one thing it does achieve is that there is generally less hyperbola here, and less all kinds of crazies. Solving larger coordination problems becomes easier.
I agree that slowing hardware gains could slow the arrival of ems, if hardware costs end up being the limiting factor. I also agree that working ems would help other AI progress. Even so, my guess is that ems come first.
In the podcast you talk about your em scenario. You mention some reason why you expect that non-em strong AI to take a long time (only 5-10% progress to date + probably slowing improvements in hardware and software in the future). But then when you discuss the em scenario, you go straight from why ems should be possible to a situation where they are very cheap. I don't get why the same factors that keep AI progress slow won't also keep em price progress slow. Slowing hardware progress seems like a big problem for ems. Also, it seems plausible to me that the insights needed to optimize em software could accelerate non-em AI progress. Do you think that the ubiquitous em scenario is the most likely scenario for the far future, or do you just think it the most interesting?
Yes sometimes there are win-win choices where you don't have to give up anything to get some things better. So just assume we've done all those obvious wins, and are now considering other choices.
Wealth wouldn't help much with invading aliens or the computer that takes over the world. But yeah, it is a handy approach when dealing with a robust economy.
>Imagine I told you to pack a bag for a trip, but I wouldn’t tell you to where. The wider the set of possibilities you needed to handle, the bigger and more expensive your bag would have to be.
You could take a bag full of money, and just buy what you need when you get there. That isn't completely robust and it isn't as efficient as knowing where you are going and packing what you will need ahead of time, but it still works pretty well. Can that strategy be generalized to dealing with the uncertainty of the future in general? I think so.
It depends on who's providing the robustness. For a single individual or organization, robustness is expensive. But aren't some social systems categorically more robust to future scenarios than others, without clearly being more expensive? A society that understood the importance of prices and complete property rights would be more robust against a slew of risky scenarios, from anti-biotic resistance to peak oil to climate change. And it would cost less than nothing. For example, Germany's flexible labour market reforms made it more robust to aggregate demand shocks. Economic growth made us more robust to weather fluctuations and drought. etc.