Monthly Archives: October 2010

Hyper-Rational Harry

As long as we on the subject of magic, let me heartily recommend my ex-co-blogger’s fanfic novel:

Eliezer Yudkowsky is writing a Harry Potter fanfic, Harry Potter and the Methods of Rationality, starring a rationalist Harry Potter with ambitions to transform the world by bringing the rationalist/scientific method to magic. (more)

I am enjoying it more than all but the first Harry Potter novels. Modern science fiction has shied away from the hard-headed rational scientist as moral hero, and it feels quite refreshing to see that genre back in a full and updated force. The book does a good job of giving basic rationality “lessons” in the context of an engaging-enough story, though the story gets a bit less engaging over time.

My main discomfort with Eliezer’s new scientist-hero genre is his beyond-Enders-Game-level over-competent hero, exceptionally moral and vastly smarter than anyone else around. This young teen hero has already mostly assimilated the wisdom of his elders and ancestors and must now mostly single-handedly solve the great mysteries of his world and fight the great evil of his age, while politely ignoring the mostly useless opinions of others. I fear that giving readers more license to imagine they are such a person mostly undoes the rationality lessons they learned. After all, much of real rationality is learning how to learn from others. But it is still a fun read.

GD Star Rating
Tagged as: , ,

Boo Dark Powers

To mark Halloween, I hereby endorse mild regulation to discourage the summoning of dark supernatural powers which might destroy us all. Seriously. Anders Sandberg:

The San Marino scale for assessing the significance of messages that could be received by aliens … is a scale from 1 to 10 based on how easily detected the transmission is, and how much information about humanity it contains. Past deliberate transmissions have managed to reach 8, ‘far reaching’. Even planetary radar manages to reach a level of 6, ‘noteworthy’.

So, how far does these [recent] advertisements go? The Deep Space Communications dish is apparently 5 meter in diameter … The Dwingeloo telescope used for the Klingon invitation is a bit larger, 25 meters, but again the transmission power is unclear. … The level 8 signal from the Arecibo used about half a million Watts of power, making it far more powerful than anything these telescopes can achieve. … It hence seems that compared to past messages these adverts are unlikely to matter: we are already transmitting other messages, these just add to the choir. …

How much should small groups of people be allowed to risk the future of humanity with low probability? Not everyone agrees that the risk from alien contact is negligible: even a very low probability times a great harm can be relevant. … Should we be equally concerned with occultists trying to summon world-changing supernatural powers? There are probably many more people today who believe in supernatural entities than mere aliens, and that some interactions with them could be harmful. Yet there are no attempts at formulating risk scales for ritual magic. … Even if we were to analyse them rationally, we need to have an ‘ultraviolet cut-off’ for the infinite number of possible-yet-exceedingly-unlikely possibilities we could worry about. How to rationally decide on this cut-off seems problematic.

Even if the risk from recent ads is only 1/1000 of the risk of other prior transmissions, since I’m not sure how big was their risk I’m reluctant to conclude that the smaller ones are “unlikely to matter.” Perhaps 1/1000 of a bigger risk is still big enough to matter. So I’d support mild regulation to discourage such transmissions, big and small.

On which risks should matter, I’d prefer to use odds from a prediction market to decide. And my guess is that they’d give non-trivial (well over one in a million) odds both that our transmissions might alert hostile aliens, and that one could “summon world-changing supernatural powers.” So I’d also support mild regulation to discourage actions that might risk our descruction by terrible supernatural dark lords. Perhaps that sounds silly, but to disagree you either have to support our destruction by dark powers, or disagree that policy should be based on market odds.

Added 1Nov: Many of you talk as if one would have to go through a phone-book length list of potential disasters before one came to this one.  But surely this is one of the top ten disasters of concern in fiction, and would be similarly high in surveys.  Yes, you might not take it seriously, but others clearly do. The question is: what probability to use in policy when people disagree on probabilities. Surely the right answer is some sort of weighed average of opinion, and while willingness to bet is a great way to weight opinions, surely most any neutral approach will give a high enough probability of disaster here to justify substantial concern.  And this justifies “mild” regulation, which looks for the easy wins of large harm reduction at low cost.  Now it might turn out that there just don’t exist any feasible mild regulations, but we should at least consider some options before drawing that conclusion.

GD Star Rating
Tagged as: , , , ,

Goertzel on Friendly AI

Ben Goertzel isn’t big on friendly AI:

SIAI’s “Scary Idea”:  … Progressing toward advanced AGI without a design for “provably non-dangerous AGI” is highly likely to lead to an involuntary end for the human race. …

Reasons for believing the Scary Idea: …

  1. If one pulled a random mind from the space of all possible minds, the odds of it being friendly to humans are very low.
  2. … If you create an AGI with a roughly-human-like value system, then this … is likely to rapidly diverge into something with little or no respect for human values.
  3. “Hard takeoffs” (in which AGIs recursively self-improve and massively increase their intelligence) are fairly likely once AGI reaches a certain level of intelligence; and humans will have little hope of stopping these events.
  4. A hard takeoff, unless it starts from an AGI designed in a “provably Friendly” way, is highly likely to lead to an AGI system that doesn’t respect the rights of humans to exist.

… I think the first of the above points is reasonably plausible, though I’m not by any means convinced. … I agree much less with the final three points listed above. …

I doubt human value is particularly fragile. Human value has evolved and … already takes multiple different forms. … I think it’s fairly robust.  … I think a hard takeoff is possible, though … I think it’s very unlikely to occur until we have an AGI system… at the level of a highly intelligent human. And I think the path to this … somewhat gradual, not extremely sudden. …

Pointing out that something scary is possible, is a very different thing from having an argument that it’s likely. The Scary Idea is certainly something to keep in mind, but there are also many other risks to keep in mind, some much more definite and palpable. …

I’m also quite unconvinced that “provably safe” AGI is even feasible. … The goal of “Friendliness to humans” or “safety” or whatever you want to call it, is rather nebulous and difficult to pin down. … One is going to need to build systems with a nontrivial degree of fundamental unpredictability. …

I think the way to come to a useful real-world understanding of AGI ethics is going to be to … study these early-stage AGI systems empirically, with a focus on their ethics as well as their cognition in the usual manner of science. … So what’s wrong with this approach?  Nothing, really — if you hold the views of most AI researchers or futurists.

I’m also not big on friendly AI, but my position differs somewhat. I’m pretty skeptical about a very local hard takeoff scenario, where within a month one unnoticed machine in a basement takes over a world like ours. And even given on such a scenario the chance that its creators could constrain it greatly via a provably friendly design seems remote. And the chance such constraint comes from a small friendliness-design team that is secretive for fear of assisting reckless others seems even more remote.

On the other hand, I think it pretty likely that growth in the world economy will speed up greatly and suddenly, that increasing intelligence in creatures will contribute to that growth, and that most future intelligence will be machine-based.  I also think it inevitable that uncontrolled evolution in a competitive world leads to future creatures with values different from ours, inducing behavior we dislike. So in this sense I see a fast takeoff to unfriendly AI as likely.

I just see little point anytime soon in trying to coordinate to prevent such an outcome. Like Ben, I think it is ok (if not ideal) if our descendants’ values deviate from ours, as ours have from our ancestors. The risks of attempting a world government anytime soon to prevent this outcome seem worse overall.

GD Star Rating
Tagged as: ,

Aliens Among Us

The regulation of our lives by large organizations is necessary for the functioning of industrial-technological society.  The result is a sense of powerlessness on the part of the average person. Unabomber

Many of our regulations apply to big firms more strongly than small firms, and and even less to homes. For example, many regulations apply only to firms with more than a certain number of employees. Now some regulations have big fixed costs of compliance, which cost more proportionally for small orgs.  But this justification doesn’t really explain that much regulation variation. For example, household hazardous waste rules let homes dispose more easily of many kinds of waste, yet trash disposal isn’t dominated by org-size fixed costs.

When I asked my enviro econ students to explain weaker home trash rules, some said firms care only for profit, while homes care about the environment, so homes don’t need rules to do the right thing. Others said the opposite, that homes rebel more against strict rules, such as by tossing trash in the woods, while firms are more obedient.

Now it seems to me that bigger orgs are in fact easier to monitor and punish, which can justify stricter rules when such rules are harder to enforce. Larger orgs regiment behavior on larger scales, making it easier to predict what one part is doing from what other parts does, and making behaviors visible to more people. For example, if one Walmart throws a certain kind of trash away illegally, its a good bet lots of other Walmarts are doing the same, and lots of employees could expose the practice.

But this is only part of the explanation. Firms obey trash rules in large part because we do random inspections of firm trash, yet would not tolerate random inspections of home trash. Big orgs are favorite movie villians, and people seem to demand higher wages to work for them. It seems we love to hate and distrust big orgs, relative to small orgs and individuals.

And this seems objectively unfair; big firms make it easier for us to monitor and discourage them from bad behavior, yet we reward this help by taxing them more, and imposing more burdens.  Big organizations are the new aliens among us, strange and suspicious to both forager and farmer eyes. We can’t look them in the eyes and feel their warmth of their empathy via ancient human protocols of understanding. Yes humans represent them, but we can see that org needs drive their actions; switch the guy at the top and they do pretty much the same things.  Big orgs display deep beyond-human intelligence we only dimly understand, and potential vast longevity.  So we suspect the worst.

Yet on the whole big orgs are a big reason we are rich and peaceful; our industrial economy depends heavily on their unmatched ability to give us what we want. Even on the uneven field in which we make them play, they keep winning, and giving us more. Pause for a moment to wonder if maybe we haven’t been just a bit unfair to the friendly alien giants among us.

GD Star Rating
Tagged as: ,

A Media Critique

This critique of political journalism can be read as a warning on too easily assuming your own lack of bias:

Prohibited from joining in political struggles, dedicated to observing what is, regardless of whether it ought to be, the savvy believe that these [journalistic] disciplines afford them a special view of the arena, cured of excess sentiment. … The savvy don’t say: I have a better argument than you. … They say: I am closer to reality than you. And more mature.” …

Writing the news so that it lands somewhere near the “halfway point between the best and the worst that might be said about someone” is not a truthtelling impulse at all. …

“He said, she said” journalism means[:] There’s a public dispute. The dispute makes news. No real attempt is made to assess clashing truth claims in the story, even though they are in some sense the reason for the story. …

The power to place certain people, causes and ideas within the deviant sphere is one of the most ideological things journalists ever do. … Journalists maintain order by either keeping the deviant out of the news entirely or identifying it within the news frame as unacceptable, radical, or just plain impossible…  If you don’t think separation of church and state is such a good idea; if you do think a single payer system is the way to go,… chances are you will never find your views reflected in the news. It’s not that there’s a one-sided debate; there’s no debate. (more)

Of course as with most critiques of journalism, this would be better directed to journalists’ customers. It is readers who drive the industry, and they aren’t especially interested in what is true, relative to what is within acceptable bounds to say.

GD Star Rating
Tagged as: ,

Supplements Kill

Curious to see what we know about the health benefits of vitamin supplements, I went looking with my assistant David Ortiz.  We found this fantastic 2007 JAMA meta-analysis of 385 publications regarding 232,000 subjects. Bottom line: on average supplements increase an ordinary adult’s chance of dying by 7%. While selenium reduces death rates by 0.2%, beta carotene increases them by 9%, vitamin C and E by 6%, and vitamin A by 20%.  Maybe you should take selinium, but mostly just stay away from supplements. From now on, I’ll avoid multivitamins. Quotes:

We searched electronic databases and bibliographies published by October 2005. All randomized trials involving adults comparing beta carotene, vitamin A, vitamin C (ascorbic acid), vitamin E, and selenium … were included. … Randomization, blinding, and follow-up were considered markers of bias in the included trials. … We included 68 randomized trials with 232 606 participants (385 publications). When all … were pooled together there was no significant effect on mortality.

Multivariate meta-regression analyses showed that low-bias risk trials (RR, 1.16; 95% CI, 1.05-1.29) and selenium (RR, 0.998; 95% CI, 0.997-0.9995) were significantly associated with mortality. In 47 low-bias trials with 180 938 participants, the antioxidant supplements significantly increased mortality (RR, 1.05; 95% CI, 1.02-1.08). In low-bias risk trials, after exclusion of selenium trials, beta carotene (RR, 1.07; 95% CI, 1.02-1.11), vitamin A (RR, 1.16; 95% CI, 1.10-1.24), and vitamin E (RR, 1.04; 95% CI, 1.01-1.07), singly or combined, significantly increased mortality. Vitamin C and selenium had no significant effect on mortality. …

The pooled effect of all supplements vs placebo or no intervention in all randomized trials was not significant (RR, 1.02; 95% CI, 0.98-1.06). … Univariate meta-regression analyses revealed significant influences of dose of beta carotene (RR, 1.004; 95% CI, 1.001-1.007; P = .012), dose of vitamin A (RR, 1.000006; 95%CI, 1.000002-1.000009; P = .003), dose of selenium (RR, 0.998; 95% CI, 0.997- 0.999; P = .002), … on mortality. None of the other covariates (dose of vitamin C; dose of vitamin E; single or combined antioxidant regimen; duration of supplementation; and primary or secondary prevention) were significantly associated with mortality. …

In trials with low-bias risk mortality was significantly increased in the supplemented group (RR, 1.05; 95% CI, 1.02-1.08). … In high-bias risk rials (low-methodological quality in >=1 of the 4 components) mortality was significantly decreased in the supplemented group (RR, 0.91; 95% CI, 0.83-1.00). …

At event rates below 1%, the Peto odds-ratio method appears to be the least biased and most powerful method when there is no substantial imbalance in treatment and control group sizes within trials. … When we applied Peto odds ratio, we found even stronger support for detrimental effects of the supplements (for all 68 trials: 1.05; 95% CI, 1.02-1.08; for the 47 low-bias risk trials: 1.07; 95% CI, 1.04-1.10; after exclusion of high-bias risk trials and selenium trials, for beta carotene: 1.09; 95% CI, 1.06-1.13; for vitamin A: 1.20; 95% CI, 1.12-1.29; for vitamin C: 1.06; 95% CI, 0.99-1.14; and for vitamin E: 1.06; 95% CI , 1.02-1.10).

Added 9:30p: I’m reminded that Phil Goetz complained the study should have tried to fit higher order moments in dose-response estimates, becuase “mean values used in the study of both A and E are in ranges known to be toxic.” Well not only is it hard to believe that authors of 385 studies chose to test dosage levels known to be deadly, it has been three years since this JAMA paper and no critic has yet bothered to redo the analysis with higher order moments, or restricted to low-dose studies. Really, for any stat analysis, you can always complain that they didn’t try some particular variation you like.  If you think some further analysis would be insightful, it should usually be up to you to try it, not up to them to try all possible variations you can imagine.

GD Star Rating
Tagged as:

Good With Bad Traits

Here’s something odd to ponder: we generally think we are better than average, but tend to think we have particular personality traits that are less socially desirable. Two from the latest JPSP:

The tendency for people to evaluate themselves more favorably than an average-peer—the better-than-average effect (BTAE)—is among the most well-documented effects in the social-psychological literature. The BTAE has been demonstrated in many populations with various methodologies. … For dimensions on which the self is positively evaluated, enhancement motives restrict the extent to which average-peer assimilation occurs. But for dimensions on which the self is negatively evaluated, enhancement motives amplify average-peer assimilation. (more)

Consensus studies from 4 cultures—in Belgium, the Czech Republic, Estonia, and Germany—as well as secondary analyses … from 29 cultures suggest that there is a cross-culturally replicable pattern of difference between internal and external perspectives for the Big Five personality traits. People see themselves as more neurotic and open to experience compared to how they are seen by other people. External observers generally hold a higher opinion of an individual’s conscientiousness than he or she does about him- or herself. As a rule, people think that they have more positive emotions and excitement seeking but much less assertiveness than it seems from the vantage point of an external observer. … A relatively strong negative correlation (r = −.53) between the average self-minus-observer profile and social desirability ratings suggests that people in most studied cultures view themselves less favorably than they are perceived by others (more)

GD Star Rating
Tagged as:

Cosmic Trash

Last week I talked on interstellar colonization at a NYC design school.  Coincidentally, just before I taped a short interview on the same subject for the Are We Alone? radio show.  I start about minute 33, and go for about six minutes.

GD Star Rating
Tagged as: ,

Improve Your Gal?

A woman often marries a man for his potential. If women married men for who they actually were, there would be far fewer marriages. When a woman loves a man, she says to herself, ‘I could improve him. Once we’re together, things will be different.’ Since I began my [couples therapy] practice in 1977, I’ve heard this refrain hundreds of times. I try to get it across to the woman that what she sees is what she gets. This is him. …

Men tend to resist change. In fact, one of the most prized characteristics of a man’s friendship with other men is total acceptance. When a woman begins to encourage a man to live up to his potential, he misunderstands that as her overall dissatisfaction with him. … The man may initially improve according to her recommendations — remember, he has a lot invested in what she thinks of him. But over time, he becomes slower to respond. …

When the marriage is on the brink of breakup, the woman drags him into my office. That’s when I hear what almost any therapist can tell you is the most repeated phrase among men: “No matter what I do, I can never please this woman.” (more)

I have many questions:  Why don’t men try as hard to improve their women?  Is it that he cares less what she will become, or would she more resist the pressure to change?  If he cares less, is this because men peak later in life, and we care more about our spouses qualities near their peak than their trough?  If she resists more, is this because example the median woman is more dominant, the median man more submissive?  Is the above pattern world-wide, or mainly in the US?

GD Star Rating
Tagged as: ,

Pity The Sex Starved

Imaging smelling good food you know that you cannot eat.  This is probably a pleasant experience, if you’ve had enough other good food to eat lately.  But it might be a painful experience if you were starving, or had long been living on a bland diet of rice and beans.

Similarly, being around attractive sexy people is often a pleasant experience, but probably feels quite different when it is clear to all that you have zero chance of attracting them, and if you feel severely deprived of satisfying sex.  And while our society is rich enough that few starve for food anymore, wealth is much less able to prevent sexual starvation.

So our society has far more sex-starved than food-starved folks.  Yet it is far more acceptable to publicly lament the plight of the food starved than the sex starved.  Signaling compassion is not about helping the needy.

GD Star Rating
Tagged as: , ,