Sam Wilson Podcast

Sam Wilson and I did a podcast for his series, on near-far, em econ, and related topics.

One topic that came up briefly deserves emphasis: robustness can be very expensive.

Imagine I told you to pack a bag for a trip, but I wouldn’t tell you to where. The wider the set of possibilities you needed to handle, the bigger and more expensive your bag would have to be. You might not need a bag at all if you knew your destination was to stay inside one of the hundred largest airports. But you’d need a big bag if you might go anywhere on the surface of the Earth. You’d need a space-suit if you might go anywhere in the solar system, and if you might go anywhere within the Sun, well we have no bag for that.

Similarly, it sounds nice to say that because the future can be hard to predict, we should seek strategies that are robust to many different futures. But the wider the space of futures one seeks to be robust against, the most expensive that gets. For example, if you insist on being ready for an alien invasion by all possible aliens, we just have no bag for that. The situation is almost as bad if you say we need to give explicit up-front-only instructions to a computer that will overnight become a super-God and take over the world.

Of course if those are the actual situations you face, then you must do your best, and pay any price, even if extinction is your most likely outcome. But you should think carefully about whether these are likely enough bag-packing destinations to make it worth being robust toward them. After all, it can be very expensive to pack a spacesuit for a beach vacation.

(There is a related formal result in learning theory: it is hard to learn anything without some expectations about the kind of world you are learning about.)

GD Star Rating
loading...
Tagged as: ,
Trackback URL:
  • http://www.abstractminutiae.tumblr.com Samuel Hammond

    It depends on who’s providing the robustness. For a single individual or organization, robustness is expensive. But aren’t some social systems categorically more robust to future scenarios than others, without clearly being more expensive? A society that understood the importance of prices and complete property rights would be more robust against a slew of risky scenarios, from anti-biotic resistance to peak oil to climate change. And it would cost less than nothing.

    • Robin Hanson

      Yes sometimes there are win-win choices where you don’t have to give up anything to get some things better. So just assume we’ve done all those obvious wins, and are now considering other choices.

    • Ari

      “A society that understood the importance of prices and complete property rights would be more robust against a slew of risky scenarios, from anti-biotic resistance to peak oil to climate change.”

      Ehm, I think societies that push a lot of norms (call if indoctrination if you will) are more robust to above-mentioned things. Let’s go from the intellectual detail to real world of politics. I’m not american but how many conservatives do you see championing for antibiotic regulation or climate change regulation? Vaccination programs? Obviously certain economic level is needed for this (which requires good enforcement of property rights).

      I’ve long hold the belief that Nordic countries are quite robust to tail risks. The public education (cultural and demographic uniformity too) pushes huge amount of things to your brain. It leads to certain kind of myopic, state-serving thinking but one thing it does achieve is that there is generally less hyperbola here, and less all kinds of crazies. Solving larger coordination problems becomes easier.

      • http://www.abstractminutiae.tumblr.com Samuel Hammond

        I agree. Norway in particular due to its sovereign wealth fund. Seems like a good strategy for those states that have the option.

  • Joe Teicher

    >Imagine I told you to pack a bag for a trip, but I wouldn’t tell you to where. The wider the set of possibilities you needed to handle, the bigger and more expensive your bag would have to be.

    You could take a bag full of money, and just buy what you need when you get there. That isn’t completely robust and it isn’t as efficient as knowing where you are going and packing what you will need ahead of time, but it still works pretty well. Can that strategy be generalized to dealing with the uncertainty of the future in general? I think so.

    • Robin Hanson

      Wealth wouldn’t help much with invading aliens or the computer that takes over the world. But yeah, it is a handy approach when dealing with a robust economy.

    • Ronfar

      There are some places where “a bag of money” wouldn’t be all that helpful, such as Antarctica, and it also leaves you vulnerable to being robbed. (Another concern I’d have is not knowing the local language.) It’s probably okay in most places, though.

  • joeteicher

    In the podcast you talk about your em scenario. You mention some reason why you expect that non-em strong AI to take a long time (only 5-10% progress to date + probably slowing improvements in hardware and software in the future). But then when you discuss the em scenario, you go straight from why ems should be possible to a situation where they are very cheap. I don’t get why the same factors that keep AI progress slow won’t also keep em price progress slow. Slowing hardware progress seems like a big problem for ems. Also, it seems plausible to me that the insights needed to optimize em software could accelerate non-em AI progress. Do you think that the ubiquitous em scenario is the most likely scenario for the far future, or do you just think it the most interesting?

    • Robin Hanson

      I agree that slowing hardware gains could slow the arrival of ems, if hardware costs end up being the limiting factor. I also agree that working ems would help other AI progress. Even so, my guess is that ems come first.

  • Dave Lindbergh

    Pack tools. More generally, this is an 80/20 type problem – for a small cost you can pack enough variety of stuff to cover most of the likely scenarios. Covering the remainder of scenarios is really difficult.

    So be reasonably prudent – pack what you can at at a reasonable cost and hope you get lucky.