Category Archives: Disaster

911 Puzzling

By Robin Hanson · April 9, 2009 7:00 pm · Comments (61)

Someone bent my ear again on 911 conspiracy theories, and I've had jigsaw-puzzle-solving fun digging through the details.  Also, I feel we should consider evidence for even pretty crazy-sounding claims when the evidence offered meets high enough standards.  To his credit physicist Steven Jones has published papers meeting such standards:

I conclude the twin towers probably held big chucks of hitech pyrotechnic materials quite uncommon in office buildings.  And a few hundred pounds of this stuff spread around the pillars of a single floor might well bring down a tower. 

BUT, I am unpersuaded by claims that plane crashes could not have induced the towers falling as they did, the sounds heard, the warnings voiced, etc. (E.g., hear him and him.)  Aside from the above findings, the match between simple theory and observation seems about as close as we should expect, given this complex and unusual situation; it would be crazy not to expect a few anomalies between simple predictions and what we saw.

Continue reading "911 Puzzling" »

GD Star Rating
loading...
Tagged as: ,

The Thing That I Protect

By Eliezer Yudkowsky · February 7, 2009 2:18 pm · Comments (19)

Followup to:  Something to Protect, Value is Fragile

"Something to Protect" discursed on the idea of wielding rationality in the service of something other than "rationality".  Not just that rationalists ought to pick out a Noble Cause as a hobby to keep them busy; but rather, that rationality itself is generated by having something that you care about more than your current ritual of cognition.

So what is it, then, that I protect?

I quite deliberately did not discuss that in "Something to Protect", leaving it only as a hanging implication.  In the unlikely event that we ever run into aliens, I don't expect their version of Bayes's Theorem to be mathematically different from ours, even if they generated it in the course of protecting different and incompatible values.  Among humans, the idiom of having "something to protect" is not bound to any one cause, and therefore, to mention my own cause in that post would have harmed its integrity.  Causes are dangerous things, whatever their true importance; I have written somewhat on this, and will write more about it.

But still – what is it, then, the thing that I protect?

Friendly AI?  No – a thousand times no – a thousand times not anymore.  It's not thinking of the AI that gives me strength to carry on even in the face of inconvenience.

Continue reading "The Thing That I Protect" »

GD Star Rating
loading...

Investing for the Long Slump

By Eliezer Yudkowsky · January 22, 2009 3:56 am · Comments (48)

I have no crystal ball with which to predict the Future, a confession that comes as a surprise to some journalists who interview me.  Still less do I think I have the ability to out-predict markets.  On every occasion when I've considered betting against a prediction market – most recently, betting against Barack Obama as President – I've been glad that I didn't.  I admit that I was concerned in advance about the recent complexity crash, but then I've been concerned about it since 1994, which isn't very good market timing.

I say all this so that no one panics when I ask:

Suppose that the whole global economy goes the way of Japan (which, by the Nikkei 225, has now lost two decades).

Suppose the global economy is still in the Long Slump in 2039.

Most market participants seem to think this scenario is extremely implausible.  Is there a simple way to bet on it at a very low price?

If most traders act as if this scenario has a probability of 1%, is there a simple bet, executable using an ordinary brokerage account, that pays off 100 to 1?

Why do I ask?  Well… in general, it seems to me that other people are not pessimistic enough; they prefer not to stare overlong or overhard into the dark; and they attach too little probability to things operating in a mode outside their past experience.

But in this particular case, the question is motivated by my thinking, "Conditioning on the proposition that the Earth as we know it is still here in 2040, what might have happened during the preceding thirty years?"

Continue reading "Investing for the Long Slump" »

GD Star Rating
loading...

Alien Bad Guy Bias

By Robin Hanson · December 25, 2008 8:30 pm · Comments (27)

The Bad Guy Bias applies to Earth signals to aliens.  From the NYT:

The makers of the new movie “The Day the Earth Stood Still” have arranged for it to be beamed into space on … the same day the movie opens here on planet Earth. … Dr. Shostak, who was a consultant for the new movie … [says] there are some people, he acknowledges, who might worry that broadcasting “The Day the Earth Stood Still” could be inimical to our interests. He added, “I think that if these people are truly worried about such things, they might best begin by shutting down the radar at the local airport.”

Shostak is right; compared to intentional signals, unintentional signals are a million times larger:

There are three large-dish instruments in the world that are currently employed for doing radar investigations of planets, asteroids and comets: ART (Arecibo Radar Telescope), GSSR (Goldstone Solar System Radar), and EPR (Evpatoria Planetary Radar). Radiating power and directional diagram of these instruments is so outstanding that it also allows us to emit radio messages to outer space, which are practically detectable everywhere in the Milky Way. This dedicated program is called METI (Messaging to Extra-Terrestrial Intelligence) …

Continue reading "Alien Bad Guy Bias" »

GD Star Rating
loading...
Tagged as: ,

You Only Live Twice

By Eliezer Yudkowsky · December 12, 2008 2:14 pm · Comments (66)
“It just so happens that your friend here is only mostly dead.  There’s a big difference between mostly dead and all dead.”
        — The Princess Bride

My co-blogger Robin and I may disagree on how fast an AI can improve itself, but we agree on an issue that seems much simpler to us than that:  At the point where the current legal and medical system gives up on a patient, they aren’t really dead.

Robin has already said much of what needs saying, but a few more points:

• Ben Best’s Cryonics FAQ, Alcor’s FAQ, Alcor FAQ for scientists, Scientists’ Open Letter on Cryonics

• I know more people who are planning to sign up for cryonics Real Soon Now than people who have actually signed up.  I expect that more people have died while cryocrastinating than have actually been cryopreserved.  If you’ve already decided this is a good idea, but you “haven’t gotten around to it”, sign up for cryonics NOW.  I mean RIGHT NOW.  Go to the website of Alcor or the Cryonics Institute and follow the instructions.

Continue reading "You Only Live Twice" »

GD Star Rating
loading...

The Bad Guy Bias

By Robin Hanson · December 9, 2008 6:00 am · Comments (26)

Shankar Vedantam:

Nations tend to focus far more time, money and attention on tragedies caused by human actions than on the tragedies that cause the greatest amount of human suffering or take the greatest toll in terms of lives. … In recent years, a large number of psychological experiments have found that when confronted by tragedy, people fall back on certain mental rules of thumb, or heuristics, to guide their moral reasoning. When a tragedy occurs, we instantly ask who or what caused it. When we find a human hand behind the tragedy — such as terrorists, in the case of the Mumbai attacks — something clicks in our minds that makes the tragedy seem worse than if it had been caused by an act of nature, disease or even human apathy. …

Tragedies, in other words, cause individuals and nations to behave a little like the detectives who populate television murder mystery shows: We spend nearly all our time on the victims of killers and rapists and very little on the victims of car accidents and smoking-related lung cancer. "We think harms of actions are much worse than harms of omission," said Jonathan Baron, a psychologist at the University of Pennsylvania. "We want to punish those who act and cause harm much more than those who do nothing and cause harm. We have more sympathy for the victims of acts rather than the victims of omission. If you ask how much should victims be compensated, [we feel] victims harmed through actions deserve higher compensation."

This bias should also afflict our future thinking, making us worry more about evil alien intent than unintentional catastrophe. 

GD Star Rating
loading...
Tagged as:

Beyond the Reach of God

By Eliezer Yudkowsky · October 4, 2008 11:42 am · Comments (160)

Followup to:  The Magnitude of His Own Folly

Today’s post is a tad gloomier than usual, as I measure such things.  It deals with a thought experiment I invented to smash my own optimism, after I realized that optimism had misled me.  Those readers sympathetic to arguments like, "It’s important to keep our biases because they help us stay happy," should consider not reading.  (Unless they have something to protect, including their own life.)

So!  Looking back on the magnitude of my own folly, I realized that at the root of it had been a disbelief in the Future’s vulnerability – a reluctance to accept that things could really turn out wrong.  Not as the result of any explicit propositional verbal belief.  More like something inside that persisted in believing, even in the face of adversity, that everything would be all right in the end.

Some would account this a virtue (zettai daijobu da yo), and others would say that it’s a thing necessary for mental health.

But we don’t live in that world.  We live in the world beyond the reach of God.

Continue reading "Beyond the Reach of God" »

GD Star Rating
loading...

The Magnitude of His Own Folly

By Eliezer Yudkowsky · September 30, 2008 7:31 am · Comments (102)

Followup to:  My Naturalistic Awakening, Above-Average AI Scientists

In the years before I met that would-be creator of Artificial General Intelligence (with a funded project) who happened to be a creationist, I would still try to argue with individual AGI wannabes.

In those days, I sort-of-succeeded in convincing one such fellow that, yes, you had to take Friendly AI into account, and no, you couldn’t just find the right fitness metric for an evolutionary algorithm.  (Previously he had been very impressed with evolutionary algorithms.)

And the one said:  Oh, woe!  Oh, alas!  What a fool I’ve been!  Through my carelessness, I almost destroyed the world!  What a villain I once was!

Now, there’s a trap I knew I better than to fall into -

- at the point where, in late 2002, I looked back to Eliezer1997‘s AI proposals and realized what they really would have done, insofar as they were coherent enough to talk about what they "really would have done".

When I finally saw the magnitude of my own folly, everything fell into place at once.  The dam against realization cracked; and the unspoken doubts that had been accumulating behind it, crashed through all together.  There wasn’t a prolonged period, or even a single moment that I remember, of wondering how I could have been so stupid.  I already knew how.

And I also knew, all at once, in the same moment of realization, that to say, I almost destroyed the world!, would have been too prideful.

It would have been too confirming of ego, too confirming of my own importance in the scheme of things, at a time when – I understood in the same moment of realization – my ego ought to be taking a major punch to the stomach.  I had been so much less than I needed to be; I had to take that punch in the stomach, not avert it.

Continue reading "The Magnitude of His Own Folly" »

GD Star Rating
loading...

Fighting a Rearguard Action Against the Truth

By Eliezer Yudkowsky · September 23, 2008 9:23 pm · Comments (7)

Followup to:  That Tiny Note of Discord, The Importance of Saying "Oops"

When we last left Eliezer2000, he was just beginning to investigate the question of how to inscribe a morality into an AI.  His reasons for doing this don’t matter at all, except insofar as they happen to historically demonstrate the importance of perfectionism.  If you practice something, you may get better at it; if you investigate something, you may find out about it; the only thing that matters is that Eliezer2000 is, in fact, focusing his full-time energies on thinking technically about AI morality; rather than, as previously, finding an justification for not spending his time this way.  In the end, this is all that turns out to matter.

But as our story begins – as the sky lightens to gray and the tip of the sun peeks over the horizon – Eliezer2001 hasn’t yet admitted that Eliezer1997 was mistaken in any important sense.  He’s just making Eliezer1997‘s strategy even better by including a contingency plan for "the unlikely event that life turns out to be meaningless"…

…which means that Eliezer2001 now has a line of retreat away from his mistake.

I don’t just mean that Eliezer2001 can say "Friendly AI is a contingency plan", rather than screaming "OOPS!"

I mean that Eliezer2001 now actually has a contingency plan.  If Eliezer2001 starts to doubt his 1997 metaethics, the Singularity has a fallback strategy, namely Friendly AI.  Eliezer2001 can question his metaethics without it signaling the end of the world.

And his gradient has been smoothed; he can admit a 10% chance of having previously been wrong, then a 20% chance.  He doesn’t have to cough out his whole mistake in one huge lump.

If you think this sounds like Eliezer2001 is too slow, I quite agree.

Continue reading "Fighting a Rearguard Action Against the Truth" »

GD Star Rating
loading...

Horrible LHC Inconsistency

By Eliezer Yudkowsky · September 21, 2008 11:12 pm · Comments (20)

Followup to: When (Not) To Use Probabilities, How Many LHC Failures Is Too Many?

While trying to answer my own question on "How Many LHC Failures Is Too Many?" I realized that I’m horrendously inconsistent with respect to my stated beliefs about disaster risks from the Large Hadron Collider.

First, I thought that stating a "one-in-a-million" probability for the Large Hadron Collider destroying the world was too high, in the sense that I would much rather run the Large Hadron Collider than press a button with a known 1/1,000,000 probability of destroying the world.

But if you asked me whether I could make one million statements of authority equal to "The Large Hadron Collider will not destroy the world", and be wrong, on average, around once, then I would have to say no.

Unknown pointed out that this turns me into a money pump.  Given a portfolio of a million existential risks to which I had assigned a "less than one in a million probability", I would rather press the button on the fixed-probability device than run a random risk from this portfolio; but would rather take any particular risk in this portfolio than press the button.

Then, I considered the question of how many mysterious failures at the LHC it would take to make me question whether it might destroy the world/universe somehow, and what this revealed about my prior probability.

If the failure probability had a known 50% probability of occurring from natural causes, like a quantum coin or some such… then I suspect that if I actually saw that coin come up heads 20 times in a row, I would feel a strong impulse to bet on it coming up heads the next time around.  (And that’s taking into account my uncertainty about whether the anthropic principle really works that way.)

Even having noticed this triple inconsistency, I’m not sure in which direction to resolve it!

(But I still maintain my resolve that the LHC is not worth expending political capital, financial capital, or our time to shut down; compared with using the same capital to worry about superhuman intelligence or nanotechnology.)

GD Star Rating
loading...