Years ago when the AI risk conversation was just starting, I was a relative skeptic, but I was part of the conversation. Since then, the conversation has become much larger, but I seem no longer part of it; it seems years since others in this convo engaged me on it.
But that's not very strong evidence that it will be particularly powerful. I mean humans can coordinate to start terrorist attacks in a pretty similar way. I mean, if you stop imagining what is normal for humans to do and just theoretically try and imagine what's the most destructive awful way humans can behave you get some pretty terrifying answers as well.
I was not aware that fusion power or space colonization had the same social opposition like genetic engineering. I thought everyone was for fusion energy. It's just a hard technical challenge.
Machine intelligence is embedded in companies today, and there's every reason to expect that situation to continue for quite a while - especially with the most powerful systems. I agree that it is true that progress is faster with the machine components, they are less limited by historical constraints, they are not intrinsically human-like, and that the combination of these factors are generally a cause for concern for humans. However, Robin may also grant these points. I think he is concerned about something a bit more specific. His headline says: "why not wait" - not "why care at all".
To further clarify re: humanlike subunits: I am *not* claiming that these guarantee a superintelligence free of monstrous behavior (as Hitler and Pol Pot’s regimes famously demonstrate), but rather that absent this property, and other things equal, the likelihood of monstrous behavior is higher.
(And, my first two reasons argue that things are *not* equal).
I am concerned about AI alignment risks mainly because, as compared with other superintelligences we have experience with (large organizations), AI capabilities *already* are increasing more rapidly, appear to have a much higher improvement ceiling, and are *not* composed of subunits inherently sharing human-like values (unlike current organizations, whose subunits are humans).
(The ‘humanlike subunit’ attribute is admittedly no guarantee of remotely reasonable behavior, but it does tend to reduce the likelihood, degree, and rapidity of onset of truly monstrous behavior. That is why its absence in the case of AI is on my list of reasons for concern).
I don't know much about the risks of being able to coordinate drones but I can believe that could be one of the ways of causing harm that are much harder to defend against than to implement.
I'm talking about "coordinating" drones only to the extent of buying 100s or 1000s of them, and leaving them in a secluded place perhaps even miles from their intended targets. Then, at a particular day and time, give commands to each of them to start up and fly to the latitude/longitude/elevation of their targets.
If that was done, and even dozens of them were discovered, there would still be no obvious clue that they were intended for particular targets, or even that they would intended to do something evil.
My main point was to push back against Robin's 'don't worry, AIs will keep each other perfectly in check' argument. I wouldn't expect those kind of competition/efficient-markets-type assumptions to apply here because the Prevent Harm AIs are at a fundamental disadvantage vs the Release Dangerous New Technologies AIs.
Yes, I think that's absolutely correct. I was taking your point even further, to point out that not only is Prevent Harm AGI going to be at a disadvantage, but plain old humans--right now and in the near future-- are going to be at a disadvantage to plain old AI (not just AGI) that exits today or in the near future, and is deployed by humans for evil purposes. And the potential damage from scenarios that I can easily envision--but don't even want to talk about publicly--is literally in the millions of lives lost range...in the U.S. alone.
I don't know much about the risks of being able to coordinate drones but I can believe that could be one of the ways of causing harm that are much harder to defend against than to implement.
"So what makes AI special in this regard?"
The 'asymmetry between destroying and preventing harm' becoming more and more of an issues applies to tech development and scientific progress in general not AI specifically.
My main point was to push back against Robin's 'don't worry, AIs will keep each other perfectly in check' argument. I wouldn't expect those kind of competition/efficient-markets-type assumptions to apply here because the Prevent Harm AIs are at a fundamental disadvantage vs the Release Dangerous New Technologies AIs.
Perhaps one important point is the asymmetry between 'destroying things' and 'repairing / preventing things being destroyed'. It's much easier to create disorder and break things than the reverse.
Yes, absolutely. This is exactly why we need to be thinking a great deal about how AI--not some future AGI, but the AI we have now, or could have in the next decade or so--can cause problems.
For example, I can think of probably hundreds of horrible things that could be done by a fleet of a couple hundred or thousand drones that can be purchased at retail outlets today. I'm talking about things that are so horrible and so relatively easy to do that I don't even want to discuss them publicly. I'm talking about things that probably would result in thousands or even millions of deaths.
So what makes AI special in this regard? It's because, with AI, one person can purchase and control so many things at so many different locations.
As Randy Newman sang in the introduction to the Monk TV show:
...I also think "shared values" like "staying alive" or also "not killing all humans" or "a working biosphere" do, uh, actually do a lot of work for us...
If you're talking about "values" that the AI share, I don't think it's safe to assume that AIs will have any values.
As I wrote elsewhere, if someone hypothetically inserted code into all the Tesla Autopilot AI to start killing pedestrians at noon on a Wednesday, I think that's exactly what the Tesla Autopilot AIs would do.
"You are imagining a single AI controlling all traffic everywhere. Imagine instead millions of AIs each controlling a little bit of traffic."
No, I'm imagining a situation not terribly different from what Tesla already does. Tesla corporate controls all the AIs of all their cars via over-the-air updates to correct software problems. It's humans changing AIs...for the *better* presumably. But I'm imagining a situation wherein someone either at Tesla or a hacker decides to do something evil...like at a particular instant in a particular day, fully acceleration 90 percent of their cars and slamming on the brakes in 10 percent of their cars.
My point is you're greatly minimizing the risk by implying that everything is OK "...once others show us sufficient liability, and liability insurance, to cover our losses in such cases."
Imagine that somehow all Teslas--let alone all vehicles on the road--suddenly crashed, due to a a malevolent on-the-air update of the Tesla AI. There would be no way that Tesla has sufficient liability protection to handle that.
One significant danger from AI is exactly as Kyle Reese warned...AI has no pain or pity or remorse. If one tried to encourage all Tesla drivers to kill as many pedestrians as they could find, virtually no Tesla drivers would do that. But Tesla AI would do it without question, if directed to do so.
Machines displacing human workers seems like a watered down version of what the more paranoid researchers are concerned about: machines neutering or forcibly displacing the entire human species. Those problems are fairly clearly related with the main difference being timescale.
I wouldn't say that comparative advantage doesn't apply - but there are various scenarios where trade between the parties fails to happen. For example, when the weaker party's position becomes unsustainable - or when the stronger party eats the weaker one. Both scenarios are on the table in the case of machine superintelligence.
Portugal 2 takes over all the resources of England in the scenario. That includes labor, land, etc. Nobody is sitting idle. All resources are under new, more efficient management, so that Portugal 2 is just as productive per hour of labor at producing wine and cloth as the original Portugal.
Look, say that you have 10 units of "infrastructure resources" to spend. You can spend 1 unit to make an England, which can produce 1/100 measures of cloth or 1/120 measures of wine per second, or some mix thereof. Or you can spend 1 unit to make a Portugal, which can produce 1/90 measures of cloth or 1/80 measures of wine per second, or some mix thereof. And let's say that you want to produce wine and cloth in some fixed ratio r = (wine production per second) / (cloth production per second), maximizing the amount of wine+cloth.
Whatever ratio you want, you're going to be better served by spending your 10 units to make 10 Portugals. If you make any Englands with your 10 units of infrastructure resources, this is always strictly worse than the solution where you replace an England with a Portugal. This despite the fact that England has a comparative advantage at producing cloth. (And if you were forced to have an England, you would have it specialize in cloth, just as comparative advantage suggests - and you'd be seeking to replace with a Portugal at the earliest opportunity.)
I didn't miss it, but that's not plausible either. Inputs being "finite" doesn't mean that they are anywhere close to the inputs formerly used by Portugal + England. If the labor population were 100x, you can bet that there would be ways to get more wine & cloth. And now the simplification of "just wine & cloth" is becoming misleading; the point is that, if all of England is idle, there must be something that that idle labor can produce, which is of more than zero value. (Hence: comparative advantage.)
There is just no way that your scenario 3 is at all a plausible economic outcome. You would have 100+ Portugals, not just 2; and England would be employed to create something, not just idle. Your scenario doesn't work.
But that's not very strong evidence that it will be particularly powerful. I mean humans can coordinate to start terrorist attacks in a pretty similar way. I mean, if you stop imagining what is normal for humans to do and just theoretically try and imagine what's the most destructive awful way humans can behave you get some pretty terrifying answers as well.
Humans and artificial intelligence have similar problems. They aren't magic super co-ordinates.
I don't think Russia favors the development of fusion energy. They have actively attempted to sabotage the deployment of nuclear power.
I was not aware that fusion power or space colonization had the same social opposition like genetic engineering. I thought everyone was for fusion energy. It's just a hard technical challenge.
Machine intelligence is embedded in companies today, and there's every reason to expect that situation to continue for quite a while - especially with the most powerful systems. I agree that it is true that progress is faster with the machine components, they are less limited by historical constraints, they are not intrinsically human-like, and that the combination of these factors are generally a cause for concern for humans. However, Robin may also grant these points. I think he is concerned about something a bit more specific. His headline says: "why not wait" - not "why care at all".
To further clarify re: humanlike subunits: I am *not* claiming that these guarantee a superintelligence free of monstrous behavior (as Hitler and Pol Pot’s regimes famously demonstrate), but rather that absent this property, and other things equal, the likelihood of monstrous behavior is higher.
(And, my first two reasons argue that things are *not* equal).
I am concerned about AI alignment risks mainly because, as compared with other superintelligences we have experience with (large organizations), AI capabilities *already* are increasing more rapidly, appear to have a much higher improvement ceiling, and are *not* composed of subunits inherently sharing human-like values (unlike current organizations, whose subunits are humans).
(The ‘humanlike subunit’ attribute is admittedly no guarantee of remotely reasonable behavior, but it does tend to reduce the likelihood, degree, and rapidity of onset of truly monstrous behavior. That is why its absence in the case of AI is on my list of reasons for concern).
I don't know much about the risks of being able to coordinate drones but I can believe that could be one of the ways of causing harm that are much harder to defend against than to implement.
I'm talking about "coordinating" drones only to the extent of buying 100s or 1000s of them, and leaving them in a secluded place perhaps even miles from their intended targets. Then, at a particular day and time, give commands to each of them to start up and fly to the latitude/longitude/elevation of their targets.
If that was done, and even dozens of them were discovered, there would still be no obvious clue that they were intended for particular targets, or even that they would intended to do something evil.
My main point was to push back against Robin's 'don't worry, AIs will keep each other perfectly in check' argument. I wouldn't expect those kind of competition/efficient-markets-type assumptions to apply here because the Prevent Harm AIs are at a fundamental disadvantage vs the Release Dangerous New Technologies AIs.
Yes, I think that's absolutely correct. I was taking your point even further, to point out that not only is Prevent Harm AGI going to be at a disadvantage, but plain old humans--right now and in the near future-- are going to be at a disadvantage to plain old AI (not just AGI) that exits today or in the near future, and is deployed by humans for evil purposes. And the potential damage from scenarios that I can easily envision--but don't even want to talk about publicly--is literally in the millions of lives lost range...in the U.S. alone.
I don't know much about the risks of being able to coordinate drones but I can believe that could be one of the ways of causing harm that are much harder to defend against than to implement.
"So what makes AI special in this regard?"
The 'asymmetry between destroying and preventing harm' becoming more and more of an issues applies to tech development and scientific progress in general not AI specifically.
My main point was to push back against Robin's 'don't worry, AIs will keep each other perfectly in check' argument. I wouldn't expect those kind of competition/efficient-markets-type assumptions to apply here because the Prevent Harm AIs are at a fundamental disadvantage vs the Release Dangerous New Technologies AIs.
Perhaps one important point is the asymmetry between 'destroying things' and 'repairing / preventing things being destroyed'. It's much easier to create disorder and break things than the reverse.
Yes, absolutely. This is exactly why we need to be thinking a great deal about how AI--not some future AGI, but the AI we have now, or could have in the next decade or so--can cause problems.
For example, I can think of probably hundreds of horrible things that could be done by a fleet of a couple hundred or thousand drones that can be purchased at retail outlets today. I'm talking about things that are so horrible and so relatively easy to do that I don't even want to discuss them publicly. I'm talking about things that probably would result in thousands or even millions of deaths.
So what makes AI special in this regard? It's because, with AI, one person can purchase and control so many things at so many different locations.
As Randy Newman sang in the introduction to the Monk TV show:
I could be wrong, now. But I don't think so.
:-(
...I also think "shared values" like "staying alive" or also "not killing all humans" or "a working biosphere" do, uh, actually do a lot of work for us...
If you're talking about "values" that the AI share, I don't think it's safe to assume that AIs will have any values.
As I wrote elsewhere, if someone hypothetically inserted code into all the Tesla Autopilot AI to start killing pedestrians at noon on a Wednesday, I think that's exactly what the Tesla Autopilot AIs would do.
"You are imagining a single AI controlling all traffic everywhere. Imagine instead millions of AIs each controlling a little bit of traffic."
No, I'm imagining a situation not terribly different from what Tesla already does. Tesla corporate controls all the AIs of all their cars via over-the-air updates to correct software problems. It's humans changing AIs...for the *better* presumably. But I'm imagining a situation wherein someone either at Tesla or a hacker decides to do something evil...like at a particular instant in a particular day, fully acceleration 90 percent of their cars and slamming on the brakes in 10 percent of their cars.
My point is you're greatly minimizing the risk by implying that everything is OK "...once others show us sufficient liability, and liability insurance, to cover our losses in such cases."
Imagine that somehow all Teslas--let alone all vehicles on the road--suddenly crashed, due to a a malevolent on-the-air update of the Tesla AI. There would be no way that Tesla has sufficient liability protection to handle that.
One significant danger from AI is exactly as Kyle Reese warned...AI has no pain or pity or remorse. If one tried to encourage all Tesla drivers to kill as many pedestrians as they could find, virtually no Tesla drivers would do that. But Tesla AI would do it without question, if directed to do so.
Machines displacing human workers seems like a watered down version of what the more paranoid researchers are concerned about: machines neutering or forcibly displacing the entire human species. Those problems are fairly clearly related with the main difference being timescale.
I wouldn't say that comparative advantage doesn't apply - but there are various scenarios where trade between the parties fails to happen. For example, when the weaker party's position becomes unsustainable - or when the stronger party eats the weaker one. Both scenarios are on the table in the case of machine superintelligence.
Portugal 2 takes over all the resources of England in the scenario. That includes labor, land, etc. Nobody is sitting idle. All resources are under new, more efficient management, so that Portugal 2 is just as productive per hour of labor at producing wine and cloth as the original Portugal.
Look, say that you have 10 units of "infrastructure resources" to spend. You can spend 1 unit to make an England, which can produce 1/100 measures of cloth or 1/120 measures of wine per second, or some mix thereof. Or you can spend 1 unit to make a Portugal, which can produce 1/90 measures of cloth or 1/80 measures of wine per second, or some mix thereof. And let's say that you want to produce wine and cloth in some fixed ratio r = (wine production per second) / (cloth production per second), maximizing the amount of wine+cloth.
Whatever ratio you want, you're going to be better served by spending your 10 units to make 10 Portugals. If you make any Englands with your 10 units of infrastructure resources, this is always strictly worse than the solution where you replace an England with a Portugal. This despite the fact that England has a comparative advantage at producing cloth. (And if you were forced to have an England, you would have it specialize in cloth, just as comparative advantage suggests - and you'd be seeking to replace with a Portugal at the earliest opportunity.)
I didn't miss it, but that's not plausible either. Inputs being "finite" doesn't mean that they are anywhere close to the inputs formerly used by Portugal + England. If the labor population were 100x, you can bet that there would be ways to get more wine & cloth. And now the simplification of "just wine & cloth" is becoming misleading; the point is that, if all of England is idle, there must be something that that idle labor can produce, which is of more than zero value. (Hence: comparative advantage.)
There is just no way that your scenario 3 is at all a plausible economic outcome. You would have 100+ Portugals, not just 2; and England would be employed to create something, not just idle. Your scenario doesn't work.