The US did, in fact, attempt to conquer Canada (war of 1812) and succeed in conquering much of Mexico (Mexican war of 1844), not to mention annexing Native American territory all around. Just before the Civil War pro-slavery forces pushed hard for the military annexation of Cuba and more of Mexico, including semi-official efforts like the Lopez expedition (1850), which attempted to recruit Robert E. Lee as its commander . The Spanish-American war (1898) was also fought heavily for territorial aggrandizement.
"US officials met with members of the National Bolivarian Armed Forces of Venezuela from 2017 to 2018 to discuss coup plans, though discussions ceased after information leaked and some of the plotters were arrested prior to their anticipated actions during the 2018 Venezuelan presidential election.[76] May 2018 presidential elections in Venezuela were boycotted by the opposition and Maduro won amid low turnout; the United States and other nations refused to recognize the elections, saying they were fraudulent.[77] National Security Advisor John Bolton said in a 1 November 2018 speech prior to the 2018 United States elections that the Trump administration would confront a "Troika of tyranny" and remove leftist governments in Cuba, Nicaragua and Venezuela;[78][66] Trump officials spoke to the media about an existing plan to overthrow Maduro, limiting oil exports to Cuba to create economic distress which would prompt its government's removal and then to finally target Nicaragua.[66][79]"
2018 does feel like a long time ago, doesn't it? ;-)
Washington's great strength as a leader was his ability to step away from power. He did it right after the revolution and then as president. He was so revered that it took 140 years for another president to think he was above giving up power.
Washington deserves credit but he was also part of a cultural tradition that opposed the idea of dictators and monarchs. The original colonies for the most part had norms of peaceful transition of power and non-hereditary leadership. So in a real sense he was bound by the expectations of the people. If he had tried to appoint himself dictator, it would have not gone unchallenged.
Conversely we saw how easy it was in China, with its long cultural tradition of emperor rulers, for Xi to appoint himself president-for-life. He was stepping into a role that is culturally expected.
By him (Xi). Not necessarily the people of China, who I think would be quite grateful for Xi to step down. As for the long tradition of emperors, dynasties ell frequently from peasant war lords, who after taking control decided not to step down, just as Lenin decided government had to wither away so he could still in power.
As for US, how many congresspersons are willing to walk away from their power? Although I'm not suggesting term limits, because we have term limits---it's called an election.
Regarding the actions of a "for-profit government" (which needs more definition; who are the shareholders for example?), I don't see it as a question of order vs chaos. It's a question of incentives. For-profit corporations are autonomous agents that maximize returns to shareholders, full stop. That's an incredibly powerful engine BUT you'd better be sure it's pointed in the right direction before you let it loose. The path to hell can be very orderly indeed.
Empirically, corporations have never handled complex goals well; look at the recent infighting at OpenAI. How to make a corporation achieve a goal like "the mutual betterment of society"? We have no experience building corporations with complex, nuanced goals like this.
The goal of making money requires for-profits to satisfy customers, and other partners. Other forms of government do not remotely seek "betterment of society", so they are not competing with that.
Democratic forms of government align the interests of the political class with the broad interests of the body politic, though? If I want to get reelected, an economic recession, spike in gas prices, massive crime wave, or other "thing people really don't like" is bad for my chances of reelection. On the other hand, low unemployment, rising wages, low gas prices, international peace and free trade, etc, are all good for my chances of reelection.
To the extent that "satisfy customers" leads to beneficent goals it seems like it would be the same, except also the "customers"/constituents who can't make the firm money don't matter.
I will repeat yesterday argument: profit is a natural guiding principle for a firm in a society where property rights are something given. In that context your profits signal your productive efficiency, and given strong property rights, profit show social value.
But for the organization whose main role is to secure property rights, what do “profits” mean? To some extent, the natural thing to maximize profits if you are the government is to reallocate property rights to yourself, and maximize profits.
Central banks do not have the objetive of profit maximization, but price stability. There is an strong reason for that… that is even bigger for government. I really like the idea of objective governance indicators, but profits? I don’t even know what it means for the government…
Property rights were extended by governments, and continue to be. They do this because it makes society as a whole richer and therefore the government richer.
I also don't think that there's good historical evidence of non-nation state rule of things like colonial territories being superior to "standard" centralized government control -- quite the opposite. There were a lot of joint-stock companies given authority over colonial areas, and many turned out to be outright scams that crippled the economies of the nations where the investors came from (Mississippi Company, South Seas Company, Darien), and the ones that actually governed didn't exactly do a sterling job running their territories (c.f. Parliamentary hearings on the British East India Company). The African territory that was the personal fief of a monarch (Belgian Congo) had it a lot worse than places which were run by a government. Etc.
If you want more recent examples, there are lots of examples of cities and states selling off portions of their infrastructure to for-profit companies (highways, parking meters, water systems) and all of the stories I hear about this are of disaster, at least in the US. While I think the evidence is against this sort of corporate control of municipal function in the US, I would believe that in other countries that have more baseline corruption in the government this could be a reasonable option.
As for "company towns", I'm not aware of any that have stood the test of time. Municipalities and states last for centuries or millennia and need to build and maintain infrastructure with a very long lifespan. Companies don't tend to exist on the same timescale.
What journalist would ever write a story about a government successfully selling off infrastructure? Without conflict, the story won’t sell papers or get clicks. So you should look askance at “all the stories [you] hear about.” They ought to be biased towards memorable disasters.
Yes, here in Reno they "sold" their parking meters and within two years they had to sever the contract and resume control. But those are examples of govt. selling things to private companies and I took Mr. Hanson to be suggesting the opposite. I spent a dozen years working on the roads in Arlington Co., Va. Potholes were fixed immediately if reported and inspectors from the county drove around every day hoping to find things that were in disrepair before they were reported. When I moved to Washoe County our govt. sells the repair work to private companies who wait until entire thoroughfares are broken down before undertaking massive projects that close major roads for upwards to two years and they pay nearly twice as much on roadwork than Arlington County spent. I am of the contrary opinion that business is more efficient than govt. Business is more efficient for the captains of the business but not for the people who need their streets repaired. But not always--you can point to the water dept. in Jackson, Ms. as a very recent example. But then that was because govt. was more or less deliberating not doing its job. And so we can argue back and forth. I don't know if there is an answer either way, but I do think Mr. Hanson has a point, if there were checks that all of the profits of govt. were returned to the citizens in fulfilled services and the overhead was kept as minimal as possible.
Dubai does (or did, not so sure since the initialed prince started running things) split its proceeds among the residents. But there is a small population and it's leaders extremely sensitive against reducing any of its control.
Adam Smith spends nearly half of Wealth of Nations arguing against the British India Co. The first Businesses were craftsmen to make things for kings or "engineers" to build buildings. Business exists only for government and by government so Mr. Hanson's idea of for-profit govt is not exactly absurd. If the profits were used not as "bribes" hoarded by the govt but returned to people as the benefits of govt.
I know the Belgian Congo was much better than Leopold's Congo Free State (and I guess he himself could argue he ended the slave trade compared to the pre-colonial status quo), but I haven't yet heard an argument that the British Raj did a better job than the British East India Company.
Hi everyone. I did enjoy Robin's historical examples given early on in the blog post. But, his conclusions about what "people" presume may be specific to people and scholars in his circle. Regarding: "people seem to presume that order is naturally rare" and "their reasoning is like that in the above cases: the usual human condition is selfish depravity, from which our current world is a fragile rare exception." Psychologists and anthropologists do not believe these things. Instead, order is not seen as a mystery, but a consequence of diverse aspects of evolved human sociality, such as evolutionary selection for cooperation (Henrich, Secret of our Success) and close attention to social norms (Gelfand, Rule breakers, Rule Makers).
Your essay is titled The Mystery of Order. My intent was to express: order is not the mystery assumed by much of your essay. Contemporary psychologists and anthropologists (and other social scientists of course) have deemed this a question that needs answering, and have provided explanations for the impressive order observed in human societies.
I agree with you Ms, Caldwell-Harris, but as I mentioned previously, for-profit govt if the profits are for the benefit of the people, then it could be a more orderly construction than for-profit corporations which are not. At issue, it seems to me, is what benefits the most and not the few and that is what govt.should do to create an orderly society. When they fail to be perceived in doing that, then the governments are challenged (disorderly). Elections are not really about "the economy, stupid"; if the economy is bad it can be seen as disorderly. But it is about the perception of order. If people feel their lives are in a crisis they vote (or act) against government; if they feel "safe",they vote for the government. And they are stats that show that that I've collected if anyone is interested.
Why do you think issues like "crime" are always popular with voters. If a party can make people feel unsafe by bringing up crime it can be a very effective tool. Whether immigrants are taking people's job doesn't need to be a reality, it merely needs to be introduced to give people a sense of unsafeness. People have always built walls to protect themselves. All walls can be breached. But people feel a wall is safe and will protect them. It is always about feeling safe more than being safe.Order is a perception even more than a reality.
I think the difference might be the quantity of search space. There have only been perhaps a few thousand human societies above a certain threshold of power/population, and their traits are heavily constrained by selection for "a society that continues to exist for more than a trivial time." If one in 10 million of those societies is disorderly enough to wipe out the others, we would be unlikely to have found that out yet. AIs are designed (and will) to start with the entire search space of possible (in this metaphor) societal traits, out to quadrillions of examples. You are correct to say that exterminative depravity in human societies is rare, (1 in 10k? 1 in 10M?) but there will be trillions of AI "societies" and they are (as you have pointed out with EMs) near-costlessly replicatable.
I would agree, and probably the next 17 million as well, with the follow up that like in all programming, the first test does not catch the important bugs.
Because, within the context of this analogy, it is powerful enough to do sufficiently destructive things (just as modern societies are powerful enough to do apocalyptic things without necessarily defeating all the other societies, but choose not to) and was "choosing" not to because its code/weights had not yet arrived at that 17 millionth configuration (perhaps we call it a software bug) that resulted in it doing so. And because that 17mth configuration is sufficiently different and also potentially more powerful precisely because it is no longer constrained by the costs of causing damage, the previous 16.99 million are unlikely to have good strategies for defeating, or mitigating the "bad" one before it (likely very quickly) can cause severe harm.
I am not saying that the bad AI will inevitably exterminate humanity, or cause doom, or defeat all the other AIs, but I am saying that your analogizing human to AI societies, and their propensity to cause harm, actually (in my opinion) leads me to be more worried about (sub-apocalyptic, but still very bad) AI harm, because they will roll the dice so many more times.
"And why don’t government treasurers now steal our money, our militaries now take over our governments, or our elected leaders now refuse to step down at term end?"
Rampant inflation, a continuous stream of money funneled to the military-industrial complex, and the existence of the deep state all suggest that the answer to these questions is that "They do precisely that!"
The American in me wants to say all kings are illegitimate tyrants, but the local ruler has better propaganda. But I recognize that is merely plausible not convincing.
So something convincing: being for profit doesn't guarantee efficacy. Boeing and IBM are ostensibly for profit, but are too big to fail, despite crashing for decades. Big tech companies like Google and Microsoft don't innovate like they did when smaller. Big companies are less efficient than small ones, and already share problems governments have. Yet for profit seems to mean buy up the competition, aqui-hire all the people driven enough to challenge you, get bigger, and safer if less efficient.
So, uhh, if you could curb those problems to the point where they are not as bad as current govt pork barrel stuff, then sure. I don't think making money sacred would make those problems better, just make us accept them. (A lot of investors already do! Do they think money is sacred?)
I do think politicians would do better to be graded on some results based metrics, not just popularity metrics that charisma shortcircuits so well.
While I disagree with you, you made me think through why, so thanks.
The default moral stance of government is evil. Governments are as evil as they think they can get away with, bearing the next election in mind. This requires painting rivals as being more evil.
I think in both cases a large counterforce has come into play (in Russia's case, the enormous financial and military aid that Ukraine has received from both NATO and the world at large, and in Israel's case they will probably completely remove Hamas from power by the year's end), to reinforce "order" and demonstrate at least part of why it seems to be an overall trend.
Namely that when you do evil things, everybody else has an incentive to gang up on you (because they don't want to be your next victim) and nobody really helps you out of the goodness of their heart (because it does not excite their sympathies). Outside the Houthis, basically the entire Muslim world is standing aside and watching as Hamas disintegrates. Russia's not getting foreign aid from China or the DPRK or Iran, but instead paying for what scraps they're getting, while the West has given Ukraine hundreds of billions of dollars of aid.
I’m not sure that the form of the corporation does enough to convince the public of their sacredness. One could imagine some kind of cap on economic returns to owners and limits on executive compensation. These might be persuasive. But (a) that would tend to disincentivize positive risk taking (moon-shots) and (b) limits like that have to be imposed through highly detailed prohibitions, which are often open to gamesmanship. So any such limit would need to be enforced by an institutional mechanism, rather than a legalistic one.
The OpenAI case is relevant here. It had a dual structure to allow it to pursue public benefit (preventing doom) and private gain. One reading of that situation is that the institutional mechanism failed when it was called on. Another is the opposite: that the ability of people to leave put constraints on abuse of the institutional mechanism. If we want to sacralize for-profit activities, we should be studying that case.
I hereby upbraid myself for not addressing co-operatives. My limited experience with co-operatives is that they seem run by insiders for insiders. That limited experience biases me against them. Further, to the extent that the insiders listen to outsiders, they tend to listen the loudest voices, presumably because they could overthrow the insiders. And they often co-opt the loud outsiders. Because the members of co-ops tend to be their customers, the loudest outsiders seem to have idiosyncratic complaints — presumably, the insiders to a decent job at satisfying the median member. So that leads to a dynamic of insider control focused at the margin on quieting idiosyncratic complainers. Over time, such a strategy seems like it would divert resources to low-value uses, endangering the co-op to the extent that it is subject to competition.
For transparency, here are the types of co-op with which I’m personally familiar:
- home owners associations
- condominium co-operatives
- electrical power distribution co-operatives
- recycling co-operatives (the only one that worked, because the members shared an overriding ideology)
I take exception with the concept that we have never tried to take over our neighbors. At least twice we tried to take Canada (still British at the time. We made several forceable efforts to get Florida. But then we successfully took over half of Mexico away from via conquest.
We're seeing some aspects of "for-profit government" emerging at the state level in the US. Oregon for example now gives a tax rebate every other year to return unspent government funds to the people. This is a far cry from a full for-profit organization, but does begin to create incentives for elected officials to use money wisely. Every voter likes to see a big rebate. The rebates are timed to coincide with election years for state governor.
The US did, in fact, attempt to conquer Canada (war of 1812) and succeed in conquering much of Mexico (Mexican war of 1844), not to mention annexing Native American territory all around. Just before the Civil War pro-slavery forces pushed hard for the military annexation of Cuba and more of Mexico, including semi-official efforts like the Lopez expedition (1850), which attempted to recruit Robert E. Lee as its commander . The Spanish-American war (1898) was also fought heavily for territorial aggrandizement.
He said “…yet it has been a long time since the U.S. has even considered conquering them.”
that was an edit made after the above comment.
Ah, apologies to Fred then.
https://en.wikipedia.org/wiki/United_States_involvement_in_regime_change_in_Latin_America
"US officials met with members of the National Bolivarian Armed Forces of Venezuela from 2017 to 2018 to discuss coup plans, though discussions ceased after information leaked and some of the plotters were arrested prior to their anticipated actions during the 2018 Venezuelan presidential election.[76] May 2018 presidential elections in Venezuela were boycotted by the opposition and Maduro won amid low turnout; the United States and other nations refused to recognize the elections, saying they were fraudulent.[77] National Security Advisor John Bolton said in a 1 November 2018 speech prior to the 2018 United States elections that the Trump administration would confront a "Troika of tyranny" and remove leftist governments in Cuba, Nicaragua and Venezuela;[78][66] Trump officials spoke to the media about an existing plan to overthrow Maduro, limiting oil exports to Cuba to create economic distress which would prompt its government's removal and then to finally target Nicaragua.[66][79]"
2018 does feel like a long time ago, doesn't it? ;-)
https://en.wikipedia.org/wiki/Monroe_Doctrine#21st-century_approaches
Washington's great strength as a leader was his ability to step away from power. He did it right after the revolution and then as president. He was so revered that it took 140 years for another president to think he was above giving up power.
Washington deserves credit but he was also part of a cultural tradition that opposed the idea of dictators and monarchs. The original colonies for the most part had norms of peaceful transition of power and non-hereditary leadership. So in a real sense he was bound by the expectations of the people. If he had tried to appoint himself dictator, it would have not gone unchallenged.
Conversely we saw how easy it was in China, with its long cultural tradition of emperor rulers, for Xi to appoint himself president-for-life. He was stepping into a role that is culturally expected.
By him (Xi). Not necessarily the people of China, who I think would be quite grateful for Xi to step down. As for the long tradition of emperors, dynasties ell frequently from peasant war lords, who after taking control decided not to step down, just as Lenin decided government had to wither away so he could still in power.
As for US, how many congresspersons are willing to walk away from their power? Although I'm not suggesting term limits, because we have term limits---it's called an election.
Regarding the actions of a "for-profit government" (which needs more definition; who are the shareholders for example?), I don't see it as a question of order vs chaos. It's a question of incentives. For-profit corporations are autonomous agents that maximize returns to shareholders, full stop. That's an incredibly powerful engine BUT you'd better be sure it's pointed in the right direction before you let it loose. The path to hell can be very orderly indeed.
Empirically, corporations have never handled complex goals well; look at the recent infighting at OpenAI. How to make a corporation achieve a goal like "the mutual betterment of society"? We have no experience building corporations with complex, nuanced goals like this.
The goal of making money requires for-profits to satisfy customers, and other partners. Other forms of government do not remotely seek "betterment of society", so they are not competing with that.
Democratic forms of government align the interests of the political class with the broad interests of the body politic, though? If I want to get reelected, an economic recession, spike in gas prices, massive crime wave, or other "thing people really don't like" is bad for my chances of reelection. On the other hand, low unemployment, rising wages, low gas prices, international peace and free trade, etc, are all good for my chances of reelection.
To the extent that "satisfy customers" leads to beneficent goals it seems like it would be the same, except also the "customers"/constituents who can't make the firm money don't matter.
Exactly Mr. Jack, exactly.
I will repeat yesterday argument: profit is a natural guiding principle for a firm in a society where property rights are something given. In that context your profits signal your productive efficiency, and given strong property rights, profit show social value.
But for the organization whose main role is to secure property rights, what do “profits” mean? To some extent, the natural thing to maximize profits if you are the government is to reallocate property rights to yourself, and maximize profits.
Central banks do not have the objetive of profit maximization, but price stability. There is an strong reason for that… that is even bigger for government. I really like the idea of objective governance indicators, but profits? I don’t even know what it means for the government…
Property rights existed long before governments, and even today are not mostly sustained by governments.
The State as (quasi) monopolist of violence is immediately the property rights allocator. Large states bequests had to be confirmed by the King:
https://reviews.history.ac.uk/review/633
So perhaps the property rights existed before the state, but after the state, they were under its control.
Property rights were extended by governments, and continue to be. They do this because it makes society as a whole richer and therefore the government richer.
You've pegged the misalignment in motivations.
I also don't think that there's good historical evidence of non-nation state rule of things like colonial territories being superior to "standard" centralized government control -- quite the opposite. There were a lot of joint-stock companies given authority over colonial areas, and many turned out to be outright scams that crippled the economies of the nations where the investors came from (Mississippi Company, South Seas Company, Darien), and the ones that actually governed didn't exactly do a sterling job running their territories (c.f. Parliamentary hearings on the British East India Company). The African territory that was the personal fief of a monarch (Belgian Congo) had it a lot worse than places which were run by a government. Etc.
You are looking at examples from centuries ago.
If you want more recent examples, there are lots of examples of cities and states selling off portions of their infrastructure to for-profit companies (highways, parking meters, water systems) and all of the stories I hear about this are of disaster, at least in the US. While I think the evidence is against this sort of corporate control of municipal function in the US, I would believe that in other countries that have more baseline corruption in the government this could be a reasonable option.
As for "company towns", I'm not aware of any that have stood the test of time. Municipalities and states last for centuries or millennia and need to build and maintain infrastructure with a very long lifespan. Companies don't tend to exist on the same timescale.
What journalist would ever write a story about a government successfully selling off infrastructure? Without conflict, the story won’t sell papers or get clicks. So you should look askance at “all the stories [you] hear about.” They ought to be biased towards memorable disasters.
Typo The Guardian isn't the only news medium in town, apologies.
The Guardian isn't the only new medium in town.
Yes, here in Reno they "sold" their parking meters and within two years they had to sever the contract and resume control. But those are examples of govt. selling things to private companies and I took Mr. Hanson to be suggesting the opposite. I spent a dozen years working on the roads in Arlington Co., Va. Potholes were fixed immediately if reported and inspectors from the county drove around every day hoping to find things that were in disrepair before they were reported. When I moved to Washoe County our govt. sells the repair work to private companies who wait until entire thoroughfares are broken down before undertaking massive projects that close major roads for upwards to two years and they pay nearly twice as much on roadwork than Arlington County spent. I am of the contrary opinion that business is more efficient than govt. Business is more efficient for the captains of the business but not for the people who need their streets repaired. But not always--you can point to the water dept. in Jackson, Ms. as a very recent example. But then that was because govt. was more or less deliberating not doing its job. And so we can argue back and forth. I don't know if there is an answer either way, but I do think Mr. Hanson has a point, if there were checks that all of the profits of govt. were returned to the citizens in fulfilled services and the overhead was kept as minimal as possible.
Has human nature changed? Or what more recent examples should we look at? (Maybe Singapore or Dubai are relevant.)
Dubai does (or did, not so sure since the initialed prince started running things) split its proceeds among the residents. But there is a small population and it's leaders extremely sensitive against reducing any of its control.
Adam Smith spends nearly half of Wealth of Nations arguing against the British India Co. The first Businesses were craftsmen to make things for kings or "engineers" to build buildings. Business exists only for government and by government so Mr. Hanson's idea of for-profit govt is not exactly absurd. If the profits were used not as "bribes" hoarded by the govt but returned to people as the benefits of govt.
I know the Belgian Congo was much better than Leopold's Congo Free State (and I guess he himself could argue he ended the slave trade compared to the pre-colonial status quo), but I haven't yet heard an argument that the British Raj did a better job than the British East India Company.
The EIC was extremely brutal and extractive at least for one period (the 1780s IIRC) and was subsequently reined in by the government.
Do we have any data to compare the two with?
Not that I know of - only historical narrative. The book I've read is "Peace, Poverty and Betrayal", though there are surely better ones.
Cowen's Law says there's academic literature on everything, but that doesn't tell us non-academics how to find it.
Hi everyone. I did enjoy Robin's historical examples given early on in the blog post. But, his conclusions about what "people" presume may be specific to people and scholars in his circle. Regarding: "people seem to presume that order is naturally rare" and "their reasoning is like that in the above cases: the usual human condition is selfish depravity, from which our current world is a fragile rare exception." Psychologists and anthropologists do not believe these things. Instead, order is not seen as a mystery, but a consequence of diverse aspects of evolved human sociality, such as evolutionary selection for cooperation (Henrich, Secret of our Success) and close attention to social norms (Gelfand, Rule breakers, Rule Makers).
Are psychologists and anthropologists thus less publicly skeptical about for-profit government?
Your essay is titled The Mystery of Order. My intent was to express: order is not the mystery assumed by much of your essay. Contemporary psychologists and anthropologists (and other social scientists of course) have deemed this a question that needs answering, and have provided explanations for the impressive order observed in human societies.
I agree with you Ms, Caldwell-Harris, but as I mentioned previously, for-profit govt if the profits are for the benefit of the people, then it could be a more orderly construction than for-profit corporations which are not. At issue, it seems to me, is what benefits the most and not the few and that is what govt.should do to create an orderly society. When they fail to be perceived in doing that, then the governments are challenged (disorderly). Elections are not really about "the economy, stupid"; if the economy is bad it can be seen as disorderly. But it is about the perception of order. If people feel their lives are in a crisis they vote (or act) against government; if they feel "safe",they vote for the government. And they are stats that show that that I've collected if anyone is interested.
Why do you think issues like "crime" are always popular with voters. If a party can make people feel unsafe by bringing up crime it can be a very effective tool. Whether immigrants are taking people's job doesn't need to be a reality, it merely needs to be introduced to give people a sense of unsafeness. People have always built walls to protect themselves. All walls can be breached. But people feel a wall is safe and will protect them. It is always about feeling safe more than being safe.Order is a perception even more than a reality.
I think he was generalizing from the sample observed in his comment section.
I think it was more than just his comment section. He's been teaching econ to students for a long time.
This feels underrated in AI discourse. What if AIs will be orderly too.
Indeed.
I think the difference might be the quantity of search space. There have only been perhaps a few thousand human societies above a certain threshold of power/population, and their traits are heavily constrained by selection for "a society that continues to exist for more than a trivial time." If one in 10 million of those societies is disorderly enough to wipe out the others, we would be unlikely to have found that out yet. AIs are designed (and will) to start with the entire search space of possible (in this metaphor) societal traits, out to quadrillions of examples. You are correct to say that exterminative depravity in human societies is rare, (1 in 10k? 1 in 10M?) but there will be trillions of AI "societies" and they are (as you have pointed out with EMs) near-costlessly replicatable.
So the FIRST AI society is unlikely to be murderous.
I would agree, and probably the next 17 million as well, with the follow up that like in all programming, the first test does not catch the important bugs.
But why would the 17 millionth AI be able to beat all the other ones that came before it?
Because, within the context of this analogy, it is powerful enough to do sufficiently destructive things (just as modern societies are powerful enough to do apocalyptic things without necessarily defeating all the other societies, but choose not to) and was "choosing" not to because its code/weights had not yet arrived at that 17 millionth configuration (perhaps we call it a software bug) that resulted in it doing so. And because that 17mth configuration is sufficiently different and also potentially more powerful precisely because it is no longer constrained by the costs of causing damage, the previous 16.99 million are unlikely to have good strategies for defeating, or mitigating the "bad" one before it (likely very quickly) can cause severe harm.
I am not saying that the bad AI will inevitably exterminate humanity, or cause doom, or defeat all the other AIs, but I am saying that your analogizing human to AI societies, and their propensity to cause harm, actually (in my opinion) leads me to be more worried about (sub-apocalyptic, but still very bad) AI harm, because they will roll the dice so many more times.
Yeah, they might actually be too orderly.
"And why don’t government treasurers now steal our money, our militaries now take over our governments, or our elected leaders now refuse to step down at term end?"
Rampant inflation, a continuous stream of money funneled to the military-industrial complex, and the existence of the deep state all suggest that the answer to these questions is that "They do precisely that!"
They do all that, but just more subtly, or at least less blatantly.
What's your thesis here? People are just naturally good to each other and don't need rules?
Sometimes we assume depravity to pump ourselves and our status quo up, sometimes it's because we've seen a lot of depravity in the status quo. http://www.threepanelsoul.com/comic/paperclip-maximizer
The American in me wants to say all kings are illegitimate tyrants, but the local ruler has better propaganda. But I recognize that is merely plausible not convincing.
So something convincing: being for profit doesn't guarantee efficacy. Boeing and IBM are ostensibly for profit, but are too big to fail, despite crashing for decades. Big tech companies like Google and Microsoft don't innovate like they did when smaller. Big companies are less efficient than small ones, and already share problems governments have. Yet for profit seems to mean buy up the competition, aqui-hire all the people driven enough to challenge you, get bigger, and safer if less efficient.
So, uhh, if you could curb those problems to the point where they are not as bad as current govt pork barrel stuff, then sure. I don't think making money sacred would make those problems better, just make us accept them. (A lot of investors already do! Do they think money is sacred?)
I do think politicians would do better to be graded on some results based metrics, not just popularity metrics that charisma shortcircuits so well.
While I disagree with you, you made me think through why, so thanks.
The default moral stance of government is evil. Governments are as evil as they think they can get away with, bearing the next election in mind. This requires painting rivals as being more evil.
Hi Robin, I'm curious what you think about Russia invading Ukraine and Hamas' actions against Israel recently? Do you count that as order?
I think in both cases a large counterforce has come into play (in Russia's case, the enormous financial and military aid that Ukraine has received from both NATO and the world at large, and in Israel's case they will probably completely remove Hamas from power by the year's end), to reinforce "order" and demonstrate at least part of why it seems to be an overall trend.
Namely that when you do evil things, everybody else has an incentive to gang up on you (because they don't want to be your next victim) and nobody really helps you out of the goodness of their heart (because it does not excite their sympathies). Outside the Houthis, basically the entire Muslim world is standing aside and watching as Hamas disintegrates. Russia's not getting foreign aid from China or the DPRK or Iran, but instead paying for what scraps they're getting, while the West has given Ukraine hundreds of billions of dollars of aid.
Perhaps benefit corporations have a role to play in sacralization of for-profit activities. https://en.m.wikipedia.org/wiki/Benefit_corporation
I’m not sure that the form of the corporation does enough to convince the public of their sacredness. One could imagine some kind of cap on economic returns to owners and limits on executive compensation. These might be persuasive. But (a) that would tend to disincentivize positive risk taking (moon-shots) and (b) limits like that have to be imposed through highly detailed prohibitions, which are often open to gamesmanship. So any such limit would need to be enforced by an institutional mechanism, rather than a legalistic one.
The OpenAI case is relevant here. It had a dual structure to allow it to pursue public benefit (preventing doom) and private gain. One reading of that situation is that the institutional mechanism failed when it was called on. Another is the opposite: that the ability of people to leave put constraints on abuse of the institutional mechanism. If we want to sacralize for-profit activities, we should be studying that case.
I hereby upbraid myself for not addressing co-operatives. My limited experience with co-operatives is that they seem run by insiders for insiders. That limited experience biases me against them. Further, to the extent that the insiders listen to outsiders, they tend to listen the loudest voices, presumably because they could overthrow the insiders. And they often co-opt the loud outsiders. Because the members of co-ops tend to be their customers, the loudest outsiders seem to have idiosyncratic complaints — presumably, the insiders to a decent job at satisfying the median member. So that leads to a dynamic of insider control focused at the margin on quieting idiosyncratic complainers. Over time, such a strategy seems like it would divert resources to low-value uses, endangering the co-op to the extent that it is subject to competition.
For transparency, here are the types of co-op with which I’m personally familiar:
- home owners associations
- condominium co-operatives
- electrical power distribution co-operatives
- recycling co-operatives (the only one that worked, because the members shared an overriding ideology)
I take exception with the concept that we have never tried to take over our neighbors. At least twice we tried to take Canada (still British at the time. We made several forceable efforts to get Florida. But then we successfully took over half of Mexico away from via conquest.
Lessons from our 96% kin.
A big chimp may overreach. But lesser males have learned to cooperate against tyranny; a clever leader with a strong partner (1).
This is how humans may have tamed themselves, turning reactive aggression into proactive, collective violence (2).
Chimp groups compete for resources. When one troop is twice as big as another, the larger will kill or drive off the smaller (3).
Men are both highly cooperative and fiercely competitive, thus achievements like the Panama Canal, and wars so frequent they must be numbered.
Self-taming is how we balanced the conflicting drives. Neither depravity nor altruism alone, suffices.
The same is true for individuals. Enough temptation may lead astray even the most upright ordinary man.
As noted in an earlier post (4), politicians are mostly irredeemable, well on their way to a separate species (5).
(1) https://www.nationalgeographic.com/animals/article/how-animals-choose-their-leaders-brute-force-democracy.
(2) https://www.theatlantic.com/magazine/archive/2019/03/how-humans-tamed-themselves/580447/.
(3) https://www.science.org/content/article/why-do-chimps-kill-each-other.
(4) https://open.substack.com/pub/overcomingbias/p/why-not-for-profit-govt?r=22y2ke&utm_campaign=comment-list-share-cta&utm_medium=web&comments=true&commentId=49713297.
(5) https://evolution.berkeley.edu/evolution-101/speciation/defining-a-species/.
We're seeing some aspects of "for-profit government" emerging at the state level in the US. Oregon for example now gives a tax rebate every other year to return unspent government funds to the people. This is a far cry from a full for-profit organization, but does begin to create incentives for elected officials to use money wisely. Every voter likes to see a big rebate. The rebates are timed to coincide with election years for state governor.