Monthly Archives: August 2008

Brief Break

By Eliezer Yudkowsky · August 31, 2008 12:00 pm · Comments (34)

I’ve been feeling burned on Overcoming Bias lately, meaning that I take too long to write my posts, which decreases the amount of recovery time, making me feel more burned, etc.

So I’m taking at most a one-week break.  I’ll post small units of rationality quotes each day, so as to not quite abandon you.  I may even post some actual writing, if I feel spontaneous, but definitely not for the next two days; I have to enforce this break upon myself.

When I get back, my schedule calls for me to finish up the Anthropomorphism sequence, and then talk about Marcus Hutter’s AIXI, which I think is the last brain-malfunction-causing subject I need to discuss.  My posts should then hopefully go back to being shorter and easier.

Hey, at least I got through over a solid year of posts without taking a vacation.

GD Star Rating
loading...

Dreams of Friendliness

By Eliezer Yudkowsky · August 30, 2008 9:20 pm · Comments (52)

Continuation of:  Qualitative Strategies of Friendliness

Yesterday I described three classes of deep problem with qualitative-physics-like strategies for building nice AIs – e.g., the AI is reinforced by smiles, and happy people smile, therefore the AI will tend to act to produce happiness.  In shallow form, three instances of the three problems would be:

  1. Ripping people’s faces off and wiring them into smiles;
  2. Building lots of tiny agents with happiness counters set to large numbers;
  3. Killing off the human species and replacing it with a form of sentient life that has no objections to being happy all day in a little jar.

And the deep forms of the problem are, roughly:

  1. A superintelligence will search out alternate causal pathways to its goals than the ones you had in mind;
  2. The boundaries of moral categories are not predictively natural entities;
  3. Strong optimization for only some humane values, does not imply a good total outcome.

But there are other ways, and deeper ways, of viewing the failure of qualitative-physics-based Friendliness strategies.

Every now and then, someone proposes the Oracle AI strategy:  "Why not just have a superintelligence that answers human questions, instead of acting autonomously in the world?"

Sounds pretty safe, doesn’t it?  What could possibly go wrong?

Continue reading "Dreams of Friendliness" »

GD Star Rating
loading...

Fake Fish

By Robin Hanson · August 30, 2008 6:00 am · Comments (20)

1/4 of sushi is mislabeled:

Two high school students … took on a freelance science project in which they checked 60 samples of seafood using a simplified genetic fingerprinting technique to see whether the fish New Yorkers buy is what they think they are getting.  They found that one-fourth of the fish samples with identifiable DNA were mislabeled.  A piece of sushi sold as the luxury treat white tuna turned out to be Mozambique tilapia, a much cheaper fish that is often raised by farming.  Roe supposedly from flying fish was actually from smelt.  Seven of nine samples that were called red snapper were mislabeled, and they turned out to be anything from Atlantic cod to Acadian redfish, an endangered species.

This is a huge fraud rate.  Will diners continue to tolerate it?  Probably, yes – I suspect diners care more about affiliating with impressive cooks and fellow diners than they do that fish is correctly labeled. 

GD Star Rating
loading...
Tagged as:

Qualitative Strategies of Friendliness

By Eliezer Yudkowsky · August 29, 2008 10:12 pm · Comments (54)

Followup to:  Magical Categories

What on Earth could someone possibly be thinking, when they propose creating a superintelligence whose behaviors are reinforced by human smiles? Tiny molecular photographs of human smiles – or if you rule that out, then faces ripped off and permanently wired into smiles – or if you rule that out, then brains stimulated into permanent maximum happiness, in whichever way results in the widest smiles…

Well, you never do know what other people are thinking, but in this case I’m willing to make a guess.  It has to do with a field of cognitive psychology called Qualitative Reasoning.

Boilwater_4

Qualitative reasoning is what you use to decide that increasing the temperature of your burner increases the rate at which your water boils, which decreases the derivative of the amount of water present. One would also add the sign of d(water) – negative, meaning that the amount of water is decreasing – and perhaps the fact that there is only a bounded amount of water.  Or we could say that turning up the burner increases the rate at which the water temperature increases, until the water temperature goes over a fixed threshold, at which point the water starts boiling, and hence decreasing in quantity… etc.

That’s qualitative reasoning, a small subfield of cognitive science and Artificial Intelligence – reasoning that doesn’t describe or predict exact quantities, but rather the signs of quantities, their derivatives, the existence of thresholds.

As usual, human common sense means we can see things by qualitative reasoning that current programs can’t – but the more interesting realization is how vital human qualitative reasoning is to our vaunted human common sense.  It’s one of the basic ways in which we comprehend the world.

Without timers you can’t figure out how long water takes to boil, your mind isn’t that precise.  But you can figure out that you should turn the burner up, rather than down, and then watch to make sure the water doesn’t all boil away.  Which is what you mainly need, in the real world.  Or at least we humans seem to get by on qualitative reasoning; we may not realize what we’re missing…

So I suspect that what went through the one’s mind, proposing the AI whose behaviors would be reinforced by human smiles, was something like this:

Continue reading "Qualitative Strategies of Friendliness" »

GD Star Rating
loading...

Cowen-Hanson Bloggingheads Topics?

By Robin Hanson · August 29, 2008 11:30 am · Comments (25)

Tyler Cowen and I, good friends who sometimes blog-spar, will tape a bloggingheads TV show this Monday.  What would folks like us to talk about? 

GD Star Rating
loading...
Tagged as:

Moral False Consensus

By Robin Hanson · August 29, 2008 6:00 am · Comments (1)

Social butterflies know less than they think about their friends’ ethics:

[Researchers] asked groups of workers and business students about ethical dilemmas … [seeking] evidence of "false consensus bias" — that is, the tendency of people to project their values and behaviors onto others.  As the size of [individual social] networks grew, so did the extent at which individuals overestimated how many others would agree with them.  Why? People discuss "safe subjects in the workplace — sports, kids, current events," the researchers wrote. So "little of the insights that people gain from social ties may apply" to moral dilemmas.

GD Star Rating
loading...
Tagged as:

Harder Choices Matter Less

By Eliezer Yudkowsky · August 28, 2008 10:02 pm · Comments (18)

…or they should, logically speaking.

Suppose you’re torn in an agonizing conflict between two choices.

Well… if you can’t decide between them, they must be around equally appealing, right?  Equally balanced pros and cons?  So the choice must matter very little – you may as well flip a coin.  The alternative is that the pros and cons aren’t equally balanced, in which case the decision should be simple.

This is a bit of a tongue-in-cheek suggestion, obviously – more appropriate for choosing from a restaurant menu than choosing a major in college.

But consider the case of choosing from a restaurant menu.  The obvious choices, like Pepsi over Coke, will take very little time.  Conversely, the choices that take the most time probably make the least difference.  If you can’t decide between the hamburger and the hot dog, you’re either close to indifferent between them, or in your current state of ignorance you’re close to indifferent between their expected utilities.

Continue reading "Harder Choices Matter Less" »

GD Star Rating
loading...

The Complexity Critique

By Robin Hanson · August 28, 2008 8:15 am · Comments (22)

Razib at Gene Expression:

I have always been struck by starkness of human hypocrisy and its incongruity in the face of avowed beliefs. … Sin is common, and human weakness in the face of contradiction the norm.  Mens’ hearts are easily divided, and simultaneously sincere in their inclinations. … All this leads to the point that I believe far too many of those of us who wish to comprehend human nature scientifically lack a basic grasp of it intuitively. … Many atheists simply lack a deep understanding of what drives people to be religious, and that our psychological model of those who believe in gods is extremely suspect. The "irrationality" and "contradiction" of human behavior may be rendered far more systematically coherent simply by adding more parameters into the model. … When I engage with these sorts of issues with readers of Overcoming Bias or Singularitarians my suspicions become even stronger because I see in some individuals an even greater lack of fluency in normal cognition than my own. … My point is that understanding human nature is not a matter of fitting humanity to our expectations and wishes, but modeling it as it is, whether one thinks that that nature is irrational or not within one’s normative framework.

This frustrating critique is frustrating common: "You’re wrong because your model is too simple.  But I’m not going to tell you what your model is missing, at least not in a clear enough way to help you improve your model."  Yes of course almost all our models are too simple.  We all know that; what we don’t know is exactly what complexities we should be adding to our models.  And for the record I was a teen cultist and my dad and brother were/are church pastors.

For social scientists I think there is actually an advantage in having a less powerful intuitive understanding of human behavior – it helps us notice things that need explaining.  To want to explain particular human behaviors you first need to see them as puzzling, and people with powerful intuitive understandings can predict behavior so well intuitively that they often don’t notice behaviors that are at odds with our best theories. 

GD Star Rating
loading...
Tagged as:

Against Modal Logics

By Eliezer Yudkowsky · August 27, 2008 6:13 pm · Comments (42)

Continuation of:  Grasping Slippery Things
Followup to:  Possibility and Could-ness, Three Fallacies of Teleology

When I try to hit a reduction problem, what usually happens is that I "bounce" – that’s what I call it.  There’s an almost tangible feel to the failure, once you abstract and generalize and recognize it.  Looking back, it seems that I managed to say most of what I had in mind for today’s post, in "Grasping Slippery Things".  The "bounce" is when you try to analyze a word like could, or a notion like possibility, and end up saying, "The set of realizable worlds [A’] that follows from an initial starting world A operated on by a set of physical laws f."  Where realizable contains the full mystery of "possible" – but you’ve made it into a basic symbol, and added some other symbols: the illusion of formality.

There are a number of reasons why I feel that modern philosophy, even analytic philosophy, has gone astray – so far astray that I simply can’t make use of their years and years of dedicated work, even when they would seem to be asking questions closely akin to mine.

The proliferation of modal logics in philosophy is a good illustration of one major reason:  Modern philosophy doesn’t enforce reductionism, or even strive for it.

Most philosophers, as one would expect from Sturgeon’s Law, are not very good.  Which means that they’re not even close to the level of competence it takes to analyze mentalistic black boxes into cognitive algorithms.  Reductionism is, in modern times, an unusual talent.  Insights on the order of Pearl et. al.’s reduction of causality or Julian Barbour’s reduction of time are rare.

So what these philosophers do instead, is "bounce" off the problem into a new modal logic:  A logic with symbols that embody the mysterious, opaque, unopened black box.  A logic with primitives like "possible" or "necessary", to mark the places where the philosopher’s brain makes an internal function call to cognitive algorithms as yet unknown.

And then they publish it and say, "Look at how precisely I have defined my language!"

Continue reading "Against Modal Logics" »

GD Star Rating
loading...

Top Teachers Ineffective

By Robin Hanson · August 27, 2008 6:00 am · Comments (32)

Yesterday I reported that top med school docs are no healthier for patients.  Today I report that even at private schools, teachers who are fully certified do not help students perform any better on math and science tests: 

Data from the National Education Longitudinal Survey of 1988 (NELS:88) were used to investigate the effect of teacher licensure status on private school students’ 12th grade math and science test scores. This data includes schooling and family background information on students that can be linked to employment information on teachers. We find that, contrary to conventional wisdom, private school students of fully certified 12th grade math and science teachers do not appear to outperform students of private school teachers who are not fully certified.

My urban econ text says:

Studies have consistently shown that graduate coursework (e.g., a Master’s degree) does not affect teacher productivity.

I expect patients are willing to pay more for top med school docs, and parents are willing to pay more for educated and certified teachers.  And I expect that this would continue even if patients and parents knew the above results.  I suspect most of the demand for teachers, doctors, and many other professionals comes from folks wanting to affiliate with certified-as-impressive people.  And merely making patients healthier or making students perform better doesn’t count much toward impressiveness, relative to academia-certified impressiveness.   

But folks don’t like to admit this directly; they’d rather pretend they care more than they do about other outputs.  Which is why folks don’t want to hear about the above results.  The media will oblige them, and so they will continue in their preferred delusions.  Bet on it. 

Added: James Hubbard points us to a related critique of MBA training.

GD Star Rating
loading...
Tagged as: