Monthly Archives: December 2008

Nonperson Predicates

By Eliezer Yudkowsky · December 26, 2008 8:47 pm · Comments (45)

Followup to:  Righting a Wrong Question, Zombies! Zombies?, A Premature Word on AI, On Doing the Impossible

There is a subproblem of Friendly AI which is so scary that I usually don't talk about it, because only a longtime reader of Overcoming Bias would react to it appropriately – that is, by saying, "Wow, that does sound like an interesting problem", instead of finding one of many subtle ways to scream and run away.

This is the problem that if you create an AI and tell it to model the world around it, it may form models of people that are people themselves.  Not necessarily the same person, but people nonetheless.

If you look up at the night sky, and see the tiny dots of light that move over days and weeks – planētoi, the Greeks called them, "wanderers" – and you try to predict the movements of those planet-dots as best you can…

Historically, humans went through a journey as long and as wandering as the planets themselves, to find an accurate model.  In the beginning, the models were things of cycles and epicycles, not much resembling the true Solar System.

But eventually we found laws of gravity, and finally built models – even if they were just on paper – that were extremely accurate so that Neptune could be deduced by looking at the unexplained perturbation of Uranus from its expected orbit.  This required moment-by-moment modeling of where a simplified version of Uranus would be, and the other known planets.  Simulation, not just abstraction.  Prediction through simplified-yet-still-detailed pointwise similarity.

Suppose you have an AI that is around human beings.  And like any Bayesian trying to explain its enivornment, the AI goes in quest of highly accurate models that predict what it sees of humans.

Models that predict/explain why people do the things they do, say the things they say, want the things they want, think the things they think, and even why people talk about "the mystery of subjective experience".

The model that most precisely predicts these facts, may well be a 'simulation' detailed enough to be a person in its own right.

Continue reading "Nonperson Predicates" »

GD Star Rating
loading...

Alien Bad Guy Bias

By Robin Hanson · December 25, 2008 8:30 pm · Comments (27)

The Bad Guy Bias applies to Earth signals to aliens.  From the NYT:

The makers of the new movie “The Day the Earth Stood Still” have arranged for it to be beamed into space on … the same day the movie opens here on planet Earth. … Dr. Shostak, who was a consultant for the new movie … [says] there are some people, he acknowledges, who might worry that broadcasting “The Day the Earth Stood Still” could be inimical to our interests. He added, “I think that if these people are truly worried about such things, they might best begin by shutting down the radar at the local airport.”

Shostak is right; compared to intentional signals, unintentional signals are a million times larger:

There are three large-dish instruments in the world that are currently employed for doing radar investigations of planets, asteroids and comets: ART (Arecibo Radar Telescope), GSSR (Goldstone Solar System Radar), and EPR (Evpatoria Planetary Radar). Radiating power and directional diagram of these instruments is so outstanding that it also allows us to emit radio messages to outer space, which are practically detectable everywhere in the Milky Way. This dedicated program is called METI (Messaging to Extra-Terrestrial Intelligence) …

Continue reading "Alien Bad Guy Bias" »

GD Star Rating
loading...
Tagged as: ,

Devil’s Offers

By Eliezer Yudkowsky · December 25, 2008 12:00 pm · Comments (33)

Previously in series:  Harmful Options

An iota of fictional evidence from The Golden Age by John C. Wright:

    Helion had leaned and said, "Son, once you go in there, the full powers and total command structures of the Rhadamanth Sophotech will be at your command.  You will be invested with godlike powers; but you will still have the passions and distempers of a merely human spirit.  There are two temptations which will threaten you.  First, you will be tempted to remove your human weaknesses by abrupt mental surgery.  The Invariants do this, and to a lesser degree, so do the White Manorials, abandoning humanity to escape from pain.  Second, you will be tempted to indulge your human weakness.  The Cacophiles do this, and to a lesser degree, so do the Black Manorials.  Our society will gladly feed every sin and vice and impulse you might have; and then stand by helplessly and watch as you destroy yourself; because the first law of the Golden Oecumene is that no peaceful activity is forbidden.  Free men may freely harm themselves, provided only that it is only themselves that they harm."
    Phaethon knew what his sire was intimating, but he did not let himself feel irritated.  Not today.  Today was the day of his majority, his emancipation; today, he could forgive even Helion's incessant, nagging fears.
    Phaethon also knew that most Rhadamanthines were not permitted to face the Noetic tests until they were octogenerians; most did not pass on their first attempt, or even their second.  Many folk were not trusted with the full powers of an adult until they reached their Centennial.  Helion, despite criticism from the other Silver-Gray branches, was permitting Phaethon to face the tests five years early…

Continue reading "Devil’s Offers" »

GD Star Rating
loading...

Harmful Options

By Eliezer Yudkowsky · December 24, 2008 9:26 pm · Comments (30)

Previously in series:  Living By Your Own Strength

Barry Schwartz's The Paradox of Choice – which I haven't read, though I've read some of the research behind it – talks about how offering people more choices can make them less happy.

A simple intuition says this shouldn't ought to happen to rational agents:  If your current choice is X, and you're offered an alternative Y that's worse than X, and you know it, you can always just go on doing X.  So a rational agent shouldn't do worse by having more options.  The more available actions you have, the more powerful you become – that's how it should ought to work.

For example, if an ideal rational agent is initially forced to take only box B in Newcomb's Problem, and is then offered the additional choice of taking both boxes A and B, the rational agent shouldn't regret having more options.  Such regret indicates that you're "fighting your own ritual of cognition" which helplessly selects the worse choice once it's offered you.

But this intuition only governs extremely idealized rationalists, or rationalists in extremely idealized situations.  Bounded rationalists can easily do worse with strictly more options, because they burn computing operations to evaluate them.  You could write an invincible chess program in one line of Python if its only legal move were the winning one.

Of course Schwartz and co. are not talking about anything so pure and innocent as the computing cost of having more choices.

If you're dealing, not with an ideal rationalist, not with a bounded rationalist, but with a human being -

Say, would you like to finish reading this post, or watch this surprising video instead?

Continue reading "Harmful Options" »

GD Star Rating
loading...

The Longshot Bias

By Robin Hanson · December 24, 2008 6:00 am · Comments (2)

I should have reported on this Snowberg-Wolfers paper long ago:

[A] longstanding empirical regularity is that betting odds provide biased estimates of the probability of a horse winning|longshots are overbet, while favorites are underbet. Neoclassical explanations focus on rational gamblers who overbet longshots due to risk-love. The competing behavioral explanations emphasize the role of misperceptions of probabilities. We provide novel empirical tests that can discriminate between these competing theories … Using a new, large-scale dataset ideally suited to implement these tests we find evidence in favor of the view that misperceptions of probability drive the favorite-longshot bias, as suggested by Prospect Theory. Along the way we provide more robust evidence on the favorite-longshot bias, falsifying the conventional wisdom that the bias is large enough to yield profit opportunities (it isn't) and that it becomes more severe in the last race (it doesn't).

Of course we implicitly underestimate odds for events of which we aren't explicitly aware.  So it is not so much that we overestimate odds for low probability events, as that we overestimate odds for low probability events to which betting markets give unusual attention.  Combinatorial betting markets should greatly reduce this problem, as every high probability event is then composed of many low probability events, all of which are available for betting.

GD Star Rating
loading...
Tagged as:

Imaginary Positions

By Eliezer Yudkowsky · December 23, 2008 12:35 pm · Comments (31)

Every now and then, one reads an article about the Singularity in which some reporter confidently asserts, "The Singularitarians, followers of Ray Kurzweil, believe that they will be uploaded into techno-heaven while the unbelievers languish behind or are extinguished by the machines."

I don't think I've ever met a single Singularity fan, Kurzweilian or otherwise, who thinks that only believers in the Singularity will go to upload heaven and everyone else will be left to rot.  Not one.  (There's a very few pseudo-Randian types who believe that only the truly selfish who accumulate lots of money will make it, but they expect e.g. me to be damned with the rest.)

But if you start out thinking that the Singularity is a loony religious meme, then it seems like Singularity believers ought to believe that they alone will be saved.  It seems like a detail that would fit the story.

This fittingness is so strong as to manufacture the conclusion without any particular observations.  And then the conclusion isn't marked as a deduction.  The reporter just thinks that they investigated the Singularity, and found some loony cultists who believe they alone will be saved.

Or so I deduce.  I haven't actually observed the inside of their minds, after all.

Has any rationalist ever advocated behaving as if all people are reasonable and fair?  I've repeatedly heard people say, "Well, it's not always smart to be rational, because other people aren't always reasonable."  What rationalist said they were?  I would deduce:  This is something that non-rationalists believe it would "fit" for us to believe, given our general blind faith in Reason.  And so their minds just add it to the knowledge pool, as though it were an observation.  (In this case I encountered yet another example recently enough to find the reference; see here.)

Continue reading "Imaginary Positions" »

GD Star Rating
loading...

Coherent Futures

By Robin Hanson · December 22, 2008 8:00 pm · Comments (33)

There is very little careful thinking about the far future.  Yes, many people talk about the far future as if they cared, but typical interest rates suggest few care much, and most far future discussion seems to be symbolic talk about current issues.  Furthermore, when people do actually directly discuss the far future, they tend to engage in an extreme far-think, vs. a more practical near-think:

[NEAR] All of these bring each other more to mind: here, now, me, us; trend-deviating likely real local events; concrete, context-dependent, unstructured, detailed, goal-irrelevant incidental features; feasible safe acts; secondary local concerns; socially close folks with unstable traits.
[FAR] Conversely, all these bring each other more to mind: there, then, them; trend-following unlikely hypothetical global events; abstract, schematic, context-freer, core, coarse, goal-related features; desirable risk-taking acts, central global symbolic concerns, confident predictions, polarized evaluations, socially distant people with stable traits.

While our future vision should fade into an increasingly vast and uncertain fog of possibilities, far future fans instead fragment into factions, each confident in a very different view of the important future issues.  Factions use such different assumptions that they rarely build on each others’ work, or even engage others in debate.  Only they really “get it” you see, and few others ever seriously consider their arguments.  Extreme far-thinking apparently produces extreme disagreement.

Such fragmentation may be acceptable when searching a large space for rare combinations, but it is severely dysfunctional for advising common actions.  We instead need to find ways for the few people who actually care about the far future to work together via a division of labor.  But how can we do that?  Just tell each faction to reconsider that they might be mistaken?

Continue reading "Coherent Futures" »

GD Star Rating
loading...
Tagged as: , ,

Rationality Quotes 20

By Eliezer Yudkowsky · December 22, 2008 5:05 pm · Comments (12)

"For every stock you buy, there is someone selling you that stock.  What is it that you know that they don't?  What is it that they know, that you don't?  Who has the edge?  If it's not you, chances are you are going to lose money on the deal."
        — Mark Cuban

"If you have two choices, choose the harder.  If you're trying to decide whether to go out running or sit home and watch TV, go running.  Probably the reason this trick works so well is that when you have two choices and one is harder, the only reason you're even considering the other is laziness.  You know in the back of your mind what's the right thing to do, and this trick merely forces you to acknowledge it."
        — Paul Graham

"Never attribute to malice that which can be adequately explained by stupidity."
        — Hanlon's Razor

"I divide my officers into four classes; the clever, the lazy, the industrious, and the stupid.  Each officer possesses at least two of these qualities.  Those who are clever and industrious are fitted for the highest staff appointments.  Use can be made of those who are stupid and lazy.  The man who is clever and lazy however is for the very highest command; he has the temperament and nerves to deal with all situations.  But whoever is stupid and industrious is a menace and must be removed immediately!"
        — General Kurt von Hammerstein-Equord

"There's no such thing as a human who doesn't commit sin.  It's not like the world is divided into sinners and the innocent.  There are only people who can and who cannot atone for their sins."
        — Ciel

"Simple stupidity is never enough.  People need to pile stupidity on stupidity on stupidity."
        — Mark C. Chu-Carroll

GD Star Rating
loading...

No Overconfidence?

By Robin Hanson · December 22, 2008 6:00 am · Comments (6)

New data question the claim that people tend to overestimate their abilities:

A large body of literature purports to find that people are generally overconfident. In particular, a better-than-average effect in which a majority of people claim to be superior to the average person has been noted for a wide range of skills, from driving, to spoken expression, to the ability to get along with others, to test taking on simple tests. The literature generally accepts that this better-than-average effect is indicative of inflated self- assessments. However, [we] recently … show that the better-than-average data … does not indicate … people have made some kind of error in their self-evaluations.  Because of this reason, almost none of the existing experimental literature on relative overconfidence can actually claim to have found overconfidence. … In this paper, we report on an experiment designed to provide a proper test of overconfidence. … As in much previous experimental work, we find a better-than-average effect among our subjects. … We find evidence that subjects are uncertain of their own types. Our experiment can be viewed as a test of the null hypothesis that people are behaving rationally (and are not overconfident). We cannot reject that hypothesis.
GD Star Rating
loading...
Tagged as:

Show-Off Bias

By Kip · December 21, 2008 8:00 pm · Comments (25)

It seems to me that self-identified smart people are biased towards complex or counter-intuitive answers to problems.  The reason is simple: complex or counter-intuitive answers allow one to show off intelligence.  So let’s call this bias “show off bias.”

Axelrod’s Tit-For-Tat may provide a good example of show off bias.  Tit-For-Tat is a simple decision algorithm for an iterated Prisoner’s Dilemma.  In deciding whether to cooperate or defect, Tit-For-Tat states: just do whatever the other person previously did.  If the other cooperated, you cooperate.  If the other defected, you defect.  Tit-For-Tat. 

The algorithm works surprisingly well.  Wikipedia tells me that “tit for tat was the most effective, winning in several annual automated tournaments against (generally far more complex) strategies created by teams of computer scientists, economists, and psychologists.”

Why didn’t these smart scientists think of Tit-For-Tat?  They probably did, or could have.  But something made Tit-For-Tat unattractive to them.  I’m suggesting that part of what made Tit-For-Tat unattractive was a smart person’s natural desire to show off.

Let me relate two other possible examples of show off bias: one well known, and one personal.  I’ll begin with the personal anecdote.

Continue reading "Show-Off Bias" »

GD Star Rating
loading...
Tagged as: ,