Monthly Archives: September 2012

Miller’s Singularity Rising

James Miller, who posted once here at OB, has a new book, Singularity Rising, out Oct 2. I’ve read an advance copy. Here are my various reactions to the book.

Miller discusses several possible paths to super-intelligence, but never says which paths he thinks likely, nor when any might happen. However, he is confident that one will happen eventually, he calls Kurzweil’s 2045 forecast “robust”, and he offers readers personal advice as if something will happen in their lifetimes.

I get a lot of coverage in chapter 13, which discusses whole brain emulations. (And Katja is mentioned on pp.213-214.) While Miller focuses mostly on what emulations imply for humans, he does note that many ems could die from poverty or obsolescence. He make no overall judgement on the scenario, however, other than to once use the word “dystopian.”

While Miller’s discussion of emulations is entirely of the scenario of a large economy containing many emulations, his discussion of non-emulation AI is entirely of the scenario of a single “ultra AI”. He never considers a single ultra emulation, nor an economy of many AIs. Nor does he explain these choices.

On ultra AIs, Miller considers only an “intelligence explosion” scenario where a human level AI turns itself into an ultra AI “in a period of weeks, days, or even hours.” His arguments for this extremely short timescale are:

  1. Self-reproducing nanotech factories might double every hour,
  2. On a scale of all possible minds, a chimp isn’t far from von Neuman in intelligence, and
  3. Evolution has trouble coordinating changes, but an AI could use brain materials and structures that evolution couldn’t.

I’ve said before that I don’t see how these imply a weeks timescale for one human level AI to make itself more powerful than the entire rest of the world put together. Miller explains my skepticism:

As Hanson told me, the implausibility of some James Bond villains illustrates a reason to be skeptical of an intelligence explosion. A few of these villains had their own private islands on which they created new powerful weapons. But weapons development is a time and resource intensive task, making it extremely unlikely that the villains small team of followers could out-innovate all of the weapons developers in the rest of the world by producing spectacularly destructive instruments that no other military force possessed. Thinking that a few henchmen, even if led by an evil genius, would do a better job at weapons development than a major defense contractor is as silly as believing that the professor on Gilligan’s Island really could have created his own coconut based technology. …

Think of an innovation race between a single AI and the entirety of mankind. For an intelligence explosion to occur, the AI has to not only win the race, but finish before humanity completes its next stride. A sufficiently smart AI could certainly do this, but an AI only a bit brighter than von Neumann would not have the slightest chance of achieving this margin of victory. (pp.215-216)

As you can tell from this quotation, Miller’s book often reads like the economics textbook he wrote. He is usually content to be a tutor, explaining common positions and intuitions behind common arguments. He does, however, explain some of his personal contributions to this field, such as his argument that preventing the destruction of the world can be a public good undersupplied by private firms, and that development might slow down just before an anticipated explosion, if investors think non-investors will gain or lose just as much as investors from the change.

I’m not sure this book has much of a chance to get very popular. The competition is fierce, Miller isn’t already famous, and while his writing quality is good, it isn’t at the popular blockbuster popular book level. But I wish his book all the success it can muster.

GD Star Rating
Tagged as: , ,

US Politics Of Medicine

After presiding over an economy with a record disappointing performance, which usually gets incumbent presidents fired, Intrade puts Obama’s chance of re-election at 79%! I attribute an important part of this to the politics of medicine. Here’s recent US medical politics in a nutshell:

Seniors vote a lot more, and they love their free medicine, so US politicians have long written them a blank check, leading to rapid cost increases. Wonks have long said “something must be done” about costs, and the left has long wanted to expand the number with health insurance. So Obama pushed through a law requiring such an expansion, and declaring an intention to do something about costs. Later. But something, they swear.

This created a vague unease among seniors that their free medicine might get cut. Vague because seniors don’t really get how exactly costs might be cut. But still, cuts! Which created an opening for a Republican to get elected president by promising to never cut senior medicine. Except that the frontrunner Republican candidate was someone who had implemented a similar program when he was governor. And then he made a “bold” move to pick a running mate with a bold plan to turn Medicare into a voucher system. Which Romney thought would give him credit for taking problems “seriously.”

Bad move. Voters don’t really like “bold” politicians. Since seniors have a better idea of what “vouchers” mean, and how exactly they lead to cuts, that let Obama more effectively attack Romney as planning to cut seniors’ free medicine. Which is sticking, because although everyone says “something must be done”, seniors don’t actually believe that their free medicine needs to be cut. So seniors in key swing states move toward Obama, and he gets re-elected.

And after the election, there’s pretty much no chance Obama will let senior medicine get cut, at least in any way they could trace back to him. Nor will the next president after him. Maybe we’ll go into more debt, or raise taxes, or cut military spending. But no way will they stop writing medical blank checks to seniors, and letting costs rise as they will.

Here’s the recent data fleshing out this public opinion story: Continue reading "US Politics Of Medicine" »

GD Star Rating
Tagged as: ,

Why Admire Brags?

A few days ago Rob Wiblin complained about our admiration of anonymous charity:

Even those who are open about their good deeds are likely to hold a special admiration for anyone they discover has been secretly helping others for years. … This norm exists because when you go on about your altruism, … perhaps you made that donation just to be able to show off your virtue and wealth to everyone else. … [But] a culture of ‘private altruism’ has some seriously perverse effects. … We are less inclined to talk about … which causes are most valuable. .. Altruistic acts … will tend to be crowded out by alternatives that are unavoidably conspicuous – impressive cars, holidays, degrees and so forth. … Someone who really cared about helping others … would want to bring up the fact whenever they could get away with it, in order to draw attention to the merits of their cause and prompt others to join in. (more)

Charity has an overt and a covert purpose. The overt purpose is to help those who can’t trade to get the help they need. To understand the covert purpose, let’s review some basics about showing that we care.

Your associates care about how helpful you are to them. Sometimes they can see very clearly how helpful you are. For example, they might see you hold a door open, or answer a direct question. But most of the time their vision is obscured. So they have to look for clues in what they can see, to infer things unseen. For example, if they see you helping a similar associate in a situation where that associate can’t see the help, they might guess that you help them in similar situations where they can’t see. Conversely if they see you make fun of someone not in the room, they might wonder if you do the same to them when they are absent.

If they see you helping someone in need who can’t much help you back, they might guess that you would similarly help them if they were in similar need, but couldn’t help you back. And if they see you helping someone in a situation where you might reasonably guess that no one could see your help, they might think you would help them in a situation where you’d guess no one could see. There is thus a close functional association, and complementarity, between charity, helping people who can’t help you back much, and anonymity, helping when the recipient and others can’t see the help.

Given this complementarity between charity and anonymity, for the purpose of signaling, it makes sense that people recommend giving anonymously, and admire folks who do so. Sure, that may end up less helping distant others in need, but we all know that we don’t care much about that.

Imagine that after one person told another “I love your new dress, it makes you look thin,” you shouted “Liar. I know you don’t like dresses like that, and anyone can see this dress doesn’t maker her look thin.” Do you think either of them would appreciate your comment? They probably both know the speaker exaggerates, but still appreciate the exchange as a signal of friendship and loyalty. You are rudely insulting them both, because they did something they admire.

You’ll seem similarly tone deaf if you point out that charity givers are not giving in ways to maximally benefit recipients. The giver and the audience both admire the gift as a signal of loyalty and caring, which they see as good things, and in addition a third party benefits from the process. Yet there you are complaining that they aren’t doing even more. They can quite reasonably see you as rude, hostile, and ungrateful. Who made you the spokesperson for the recipients of their charity? Don’t you see how white lies smooth the social fabric?

GD Star Rating
Tagged as: , ,

If elections aren’t a Pascal’s mugging, existential risk shouldn’t be either

A response I often hear to the idea of dedicating one’s life to reducing existential risk, or increasing the likelihood of a friendly artificial general intelligence, is that it represents a form of ‘Pascal’s mugging’, a problem memorably described in a dialogue by Nick Bostrom. Because of the absurd conclusion of the Pascal’s mugging case, some people have decided not to trust expected value calculations when thinking about about extremely small likelihoods of enormous payoffs.

While there are legitimate question marks over whether existential risk reduction really does offer a very high expected value, and we should correct for ‘regression to the mean‘, cognitive biases and so on, I don’t think we have any reason to discard these calculations altogether. The impulse to do so seems mostly driven by a desire to avoid the weirdness of the conclusion, rather than actually having a sound reason to doubt it.

A similar activity which nobody objects to on such theoretical grounds is voting, or political campaigning. Considering the difference in vote totals and the number of active campaigners, the probability that someone volunteering for a US presidential campaign will swing the outcome seems somewhere between 1 in 100,000 and 1 in 10,000,000. The US political system throws up significantly different candidates for a position with a great deal of power over global problems. If a campaigner does swing the outcome, they can therefore have a very large and positive impact on the world, at least in subjective expected value terms.

While people may doubt the expected value of joining such a campaign on the grounds that the difference between the candidates isn’t big enough, or the probability of changing the outcome too small, I have never heard anyone say that the ‘low probability, high payoff’ combination means that we must dismiss it out of hand.

What is the probability that a talented individual could averting a major global catastrophic risk if they dedicated their life to it? My guess is it’s only an order of magnitude or two lower than a campaigner swinging an election outcome. You may think this is wrong, but if so, imagine that it’s reasonable for the sake of keeping this blog post short. How large is the payoff? I would guess many many orders of magnitude larger than swinging any election. For that reason it’s a more valuable project in total expected benefit, though also one with a higher variance.

To be sure, the probability and payoff are now very small and very large numbers respectively, as far as ordinary human experience goes, but they remain far away from the limits of zero and infinity. At what point between the voting example, and the existential risk reduction example, should we stop trusting expected value? I don’t see one.

Building in some arbitrary low probability, high payoff ‘mugging prevention’ threshold would lead to the peculiar possibility that for any given project, an individual with probability x of a giant payout could be advised to avoid it, while a group of 100 people contemplating the same project, facing a probability ~100*x of achieving the same payoff could be advised to go for it. Now that seems weird to me. We need a better solution to Pascal’s mugging than that.

GD Star Rating
Tagged as: , , , ,

Rude research

Bryan Caplan says intelligence research is very unpopular because it looks so bad to call half of people stupider than average, let alone stupid outright. Calling people stupid is rude.

But if this is the main thing going on, many other kinds of research should be similarly hated. It’s rude to call people lazy, ugly bastards whose mothers wouldn’t love them. Yet there is little hostility regarding research into conscientiousness, physical attractiveness, parental marriage status, or personal relationships. At least as far as I can tell. Is there? Or what else is going on with intelligence?


GD Star Rating
Tagged as: , ,

Covert virtue – the signal that doesn’t bark?

An attitude I often come across is that if you do a virtuous thing, it is impolite to blow your own trumpet about it. ‘Give privately!’ is the catchcry. Even those who are open about their good deeds are likely to hold a special admiration for anyone they discover has been secretly helping others for years, and never even mentioned it.

I asked around and apparently this norm exists because when you go on about your altruism, it calls into question  your motivation. Perhaps you made that donation just to be able to show off your virtue and wealth to everyone else, rather than being motivated by ‘pure’ compassion. Someone who only cared about helping others would apparently keep their hands busy, and their mouth shut.

I think the reality is the complete reverse. A culture of ‘private altruism’ has some seriously perverse effects, and anyone who really cares about doing good in the world should be working to undermine it.

Firstly, it means we are less inclined to talk about and share the information we have about which causes are most valuable and effective. Given that donations to charity and other approaches to making the world a better place vary in cost effectiveness across many orders of magnitude, this is a huge loss.

Secondly, if people can’t gain social acceptance from altruistic acts, those acts will tend to be crowded out by alternatives that are unavoidably conspicuous – impressive cars, holidays, degrees and so forth – that will do a better job of signalling how rich, noble and interesting they are. On top of this, people will become biased towards conspicuous but ineffective ways of helping others. It’s easy to keep a (very valuable!) bank transfer secret, and pretty gauche to post the receipt on social media sites. But flying to Africa to help build a school, or signing up to a Facebook group? Everyone will find out about that! Sadly, signalling ‘arms races’ over conspicuous consumption and slacktivism, rather than ‘effective altruism’, are exactly what I observe around me.

In light of this, private giving, far from being consistent with a pure and virtuous motivation, is actually deeply suspicious. Someone who really cared about helping others as much as possible, and was making substantial sacrifices to do so, would want to bring up the fact whenever they could get away with it, in order to draw attention to the merits of their cause and prompt others to join in. Those who ‘give privately’, must care more about blindly following harmful social norms – or more likely, getting extra admiration for their deeds when people ‘accidentally’ discover what they have secretly been up to. This could be a ‘signal that doesn’t bark.’

So next time you do something good, find a way to shout it from the rooftops, especially if the act is particularly big, valuable or easy to do discreetly. If anyone tries to call you out for ‘showing off’, politely explain why the pure of heart have no choice.

Update: Here’s an amusing video on the topic. (HT David Barry)

GD Star Rating
Tagged as: , ,

USA, Beware 2020

Both Nature and New Scientist recently covered the work of Peter Turchin, who suggests, based on prior trends, that the US is in for a new period of political instability peaking around 2020. He finds that historically US instability has peaked about every fifty years:

He also found this 50 years cycle in Roman and French history, but not in Chinese history. This evidence seems sufficient to mildly raise my expectation of instability at that time, relative to what I would have otherwise thought. Turchin also sees a 150 year cycle in six (de-trended) parameters that suggest instability:

This suggests a US peak in the decades surrounding 2040. Other civilizations have had such long waves, but with widely varying periods. This also mildly raises my expectation of instability in that period.

Even so, the strongest trend we see is a long term worldwide decline in such things. So my strongest expectation is for a continued long term decline in instability. But yes, let’s watch out for the US in 2020.

GD Star Rating
Tagged as: , ,

Significance and motivation

Over at philosophical disquisitions, John Danaher is discussing Aaron Smuts’ response to Bernard Williams’ argument that immortality would be tedious. Smuts’ thesis, in Danaher’s words, is a familiar one:

Immortality would lead to a general motivational collapse because it would sap all our decisions of significance.

This is interestingly at odds with my observations, which suggests that people are much more motivated to do things that seem unimportant, and have to constantly press themselves to do important things once in a while. Most people have arbitrary energy for reading unimportant online articles, playing computer games, and talking aimlessly. Important articles, serious decisions, and momentous conversations get put off.

Unsurprisingly then, people also seem to take more joy from apparently long-run insignificant events. Actually I thought this was the whole point of such events. For instance people seem to quite like cuddling and lazing in the sun and eating and bathing and watching movies. If one had any capacity to get bored of these things, I predict it would happen within the first century. While significant events also bring joy, they seem to involve a lot more drudgery in preceding build up.

So it seems to me that living forever could only take the pressure off and make people more motivated and happy. Except inasmuch as the argument is faulty in other ways, e.g. impending death is not the only time constraint on activities.

Have I missed something?

GD Star Rating
Tagged as: , , , ,

OB/LW Party, Berkeley

Robin and I will both be in the Bay Area early for the Singularity Summit. We’d like to warm up for the big weekend with a little Overcoming Bias/LessWrong meetup party. The folks of 2135 Oregon St have kindly lent their home for this purpose, so please join us there between 7 and 10pm on Thurs 11 October to practice chatting about important and interesting things and maybe dance a bit. There’ll be some snacks and drinks, but feel free to bring more. There’ll be street parking and Ashby Bart station half a mile away. Hopefully there’ll be Robin dancing.

GD Star Rating

Inequality /=> Revolt

Famous historical revolutions were not consistently caused by high or rising income inequality:

[French income] inequality during the eighteenth century was large but decreased during the revolutionary period (1790-1815). … When industrialisation began about 1830, inequality increased until sometime in the 1860s. (more)

In 1904, on the eve of military defeat and the 1905 Revolution, Russian income inequality was middling by the standards of that era, and less severe than inequality has become today in such countries as China, the United States, and Russia itself. (more)

In 1774 the American colonies had average incomes exceeding those of the Mother Country, even when slave households are included in the aggregate. … American colonists had much more equal incomes than did households in England and Wales around 1774. Indeed, New England and the Middle Colonies appear to have been more egalitarian than anywhere else in the measureable world. Income inequality rose dramatically between 1774 and 1860, especially in the South. (more)

So why do most people so confidently believe that revolutions were caused by high or rising inequality? I’d guess its because it feels like a nice way to affirm your support for the standard forager value of more equality.

Added 24Sept: OK, I see that the French data isn’t so relevant to my point.

GD Star Rating
Tagged as: , , ,