Tag Archives: Disagreement

Caplan Audits Age of Em

When I showed Bryan Caplan an early draft of my book, his main concern was that I didn’t focus enough on humans, as he doesn’t think robots can be conscious. In his first critical post, he focused mainly on language and emphasis issues. But he summarized “the reasoning simply isn’t very rigorous”, and he gave 3 substantive objections:

The idea that the global economy will start doubling on a monthly basis is .. a claim with a near-zero prior probability. ..

Why wouldn’t ems’ creators use the threat of `physical hunger, exhaustion, pain, sickness, grime, hard labor, or sudden unexpected death’ to motivate the ems? .. `torturing’ ems, .. why not?” ..

Why wouldn’t ems largely be copies of the most “robot-like” humans – humble workaholics with minimal personal life, content to selflessly and uncomplainingly serve their employers?

He asked me direct questions on my moral evaluation of ems, so I asked him to estimate my overall book accuracy relative to the standard of academic consensus theories, given my assumptions. Caplan said:

The entire analysis hinges on which people get emulated, and there is absolutely no simple standard academic theory of that. If, as I’ve argued, we would copy the most robot-like people and treat them as slaves, at least 90% of Robin’s details are wrong.

Since I didn’t think how docile are ems matters that much for most of my book, I challenged him to check five random pages. Today, he reports back:

Limiting myself to his chapters on Economics, Organization, and Sociology, [half of the book’s six sections] .. After performing this exercise, I’m more inclined to say Robin’s only 80% wrong. .. My main complaint is that his premises about em motivation are implausible and crucial.

Caplan picked 23 quotes from those pages. (I don’t know how picked; I count ~35 claims.) In one of these (#22) he disputes the proper use of the word “participate”, and in one (#12) he says he can’t judge.

In two more, he seems to just misread the quotes. In #21, I say taxes can’t discourage work by retired humans, and he says but ems work. In #8 I say if most ems are in the few biggest cities, they must also be in the few biggest nations (by population). He says there isn’t time for nations to merge.

If I set aside all these, that leaves 19 evaluations, out of which I count 7 (#1,4,9,13,17,19,20) where he says agree or okay, making me only 63% wrong in his eyes. Now lets go through the 12 disagreements, which fall into five clumps.

In #6, Caplan disagrees with my claim that “well-designed computers can be secure from theft, assault, and disease.” On page 62, I had explained:

Ems may use technologies such as provably secure operating system kernels (Klein et al. 2014), and capability-based secure computing systems, which limit the powers of subsystems (Miller et al. 2003).

In #5, I had cited sources showing that in the past most innovation has come from many small innovations, instead of a few big ones. So I said we should expect that for ems too. Caplan says that should reverse because ems are more homogenous than humans. I have no idea what he is thinking here.

In #3,7, he disagrees with my applying very standard urban econ to ems:

It’s not clear what even counts as urban concentration in the relevant sense. .. Telecommuting hasn’t done much .. why think ems will lead to “much larger” em cities? .. Doesn’t being a virtual being vitiate most of the social reasons to live near others? ..

But em virtual reality makes “telecommuting” a nearly perfect substitute for in-person meetings, at least at close distances. And one page before, I had explained that “fast ems .. can suffer noticeable communication delays with city scale separations.” In addition, many ems (perhaps 20%) do physical tasks, and all are housed in hardware needing physical support.

In #2,23, Caplan disagrees with my estimating that the human fraction of income controlled slowly falls, because he says all ems must always remain absolute slaves; “humans hold 100% of wealth regardless .. ems own nothing.”

Finally, half of his disagreements (#10,11,14,15,16,18) stem from his seeing ems them as quite literally “robot-like”. If not for this, he’d score me as only 31% wrong. According to Caplan, ems are not disturbed by “life events”, only by disappointing their masters. They only group, identify, and organize as commanded, not as they prefer or choose. They have no personality “in a human sense.” They never disagree with each other, and never need to make excuses for anything.

Remember, Caplan and I agree that the key driving factor here is that a competitive em world seeks the most productive (per subjective minute) combinations of humans to scan, mental tweaks and training methods to apply, and work habits and organization to use. So our best data should be the most productive people in the world today, or that we’ve seen in history. Yet the most productive people I know are not remotely “robot-like”, at least in the sense he describes above. Can Caplan name any specific workers, or groups, he knows that fit the bill?

In writing the book I searched for literatures on work productivity, and used many dozens of articles on specific productivity correlates. But I never came across anything remotely claiming “robot-like” workers (or tortured slaves) to be the most productive in modern jobs. Remember that the scoring standard I set was not personal intuition but the consensus of the academic literature. I’ve cited many sources, but Caplan has yet to cite any.

From Caplan, I humbly request some supporting citations. But I think he and I will make only limited progress in this discussion until some other professional economists weigh in. What incantations will summon the better spirits of the Econ blogosphere?

GD Star Rating
loading...
Tagged as: ,

Guardian on Age of Em

Age of Em is the “book of the day” today at the Guardian newspaper, the 5th most widely read one in the world. Reviewer Steven Poole hates the em world:

The Age of Em is a fanatically serious attempt .. to use economic and social science to forecast in fine detail how this world (if it is even possible) will actually work. The future it portrays is very strange and, in the end, quite horrific for everyone involved. .. This hellish cyberworld is quite cool to think about in a dystopian Matrixy way, although the book is much drier than fiction.

I’m fine with people not liking the em world, if they understand it. But disliking the world also seems to translate into disliking my analysis. My point by point responses:

Hanson says it reads more like an encyclopedia. But if it’s an encyclopedia, what are its sources?

References take 31 pages, others have complained of too many cites, and you complain of dry text. Yet you really wanted more cites & references?

“Today,” he complains, “we take far more effort to study the past than the future, even though we can’t change the past.” Yes, you might respond: that is because we literally cannot “study” the future – because either it doesn’t exist or (in the block-universe model of time) it does exist but is completely inaccessible to us.

We infer theories from data on the present and past. The whole reason for theory is to help us infer things where we don’t have data. Like the future. That is what theorists do. So we can study the future by applying our best theories, as I tried to do in the book.

Given that, the book’s confidence in its own brilliantly weird extrapolations is both impressive and quite peculiar. Hanson describes his approach as that of “using basic social theory, in addition to common sense and trend projection, to forecast future societies”. The casual use of “common sense” there should, as always, ring alarm bells. And a lot of the book’s sense is arguably quite uncommon.

Here you insinuate that much is wrong, but you don’t actually point out anything specific as wrong.

The governing tone is strikingly misanthropic, despairing of current humans’ “maladaptation” to the environment.

How is it remotely “hating” of people to see recent behavior as more evolutionarily maladaptive?

And there is an unargued assumption throughout that social patterns and institutions are more likely to revert to pre-industrial norms in the future.

I argue explicitly in some detail for some attitudes reverting to those more typical of poor farmers, when ems get poor. But the only institutions that might revert would be those driven mainly by attitudes, such as perhaps democracy.

Hanson .. erects a large edifice of sociological speculation on how the liberal use of em copies and backups will change attitudes to sex, law, death and pretty much everything else. But .. if someone announces they will upload my consciousness into a robot and then destroy my existing body, I will take this as a threat of murder. .. So ems – the first of whom are, by definition, going to have minds identical to those of humans – may very well exhibit the same kind of reaction, in which case a lot of Hanson’s more thrillingly bizarre social developments will not happen.

Yes, you feel strongly, but everyone need not share your feelings. Yes, the first brain scans will be destructive, but out of a world population of billions it only takes a few biological humans willing to be scanned this way to fill the em world. And if there were only a few of them, they’d each earn trillions.

But then, the rather underwhelming upshot of this project is that fast-living and super-clever ems will probably crack the problem of proper AI – actual intelligent machines – within a year or so of ordinary human time.

I didn’t say “probably” here; I gave that as one identifiable possibility.

Given that this future is so gloomy for just about everyone, one does end up wondering why Hanson wants to wake up in it – he reveals in the book that he has arranged to be cryogenically frozen on his death. I suppose it is at least possible that, one day, he could open his eyes and have the last laugh, as he surveys the appalling future he foresaw so long ago.

Because I describe a world you don’t like I must be a people hater pleased to see everyone suffer? Really?! For the record, I don’t now see the em world as appalling, and if I changed my mind on that upon seeing it up close, I’d be quite disappointed.

GD Star Rating
loading...
Tagged as: ,

Caplan on Age of Em

As many have noted, ours in an era of ideological polarization. On topics where there are strong emotions, we tend to gravitate to extremes, and are less interest in intermediate positions. Which is a problem for my book; while most see it as too weird, others mostly see it as not weird enough. A  tech futurism minority expects to soon see very rapid progress in artificial intelligence and machine learning, and so see brain emulations as too slow and inefficient compared to the super-intelligence they foresee. And the majority to whom that seems pretty crazy also seem brain emulations as similarly crazy; they don’t care much if ems seem a bit less crazy.

At least future tech enthusiasts who think my book not weird enough are willing to write reviews to say so. But those who think my book too weird mostly stay silent; I’ve heard privately of many who were going to cover the book before they fully realized what it is about. So I thank my college Bryan Caplan for being willing to say what others won’t, in his critical review. His review is long, with ten criticisms. This response will also be long, going point by point.

Six of his ten objections seem to be mainly about my language. (His review is indented, and often contains book quotes; my replies are not.) Continue reading "Caplan on Age of Em" »

GD Star Rating
loading...
Tagged as: ,

Age of Em Criticism

My book’s topic seems to me so obviously important that I figure a reader’s main question must be whether he can trust me to actually know something on it. As a result, potential readers should be especially interested to hear criticisms; where do reviewers think my book gets it wrong? And as the book draws on many disciplines, readers should be especially interested in expert criticism, i.e., reviewers who find fault in an area they know well. Let us consider the reviews so far.

Three reviews so far can be seen as “main stream media.” At the Financial Times, journalist Sarah O’Connor calls the book “alluring” and “fascinating”, but notes that not everyone will accept the premise that ems are possible or “that current economic and social theories will hold in this strange new world.” However, the closest she gets to direct criticism is:

Some of the forecasts seem old-fashioned, like the notion that male ems will prefer females with “signs of nurturing inclinations and fertility, such as youthful good looks” while females will prefer males with “signs of wealth and status”.

At the Guardian, journalist Zoe Williams uses the book to direct readers to her critical question: “In a world without work, how do we distribute resources?” At Reason, journalist Ronald Bailey calls the book “fascinating”, and summarizes it in detail, but doesn’t otherwise evaluate it, other than to note that “other futurists have projected other pathways” that the future might take.

There are 2.5 reviews at widely read blogs. Economist Tyler Cowen likes the book, but cares less about its official topic than its indirect uses, such as a “Straussian commentary on the world we actually live in” and “A reminder of how strange everything is.” Economist Bryan Caplan has posted half of a review, on “What’s Right in Robin Hanson’s The Age of Em”; his other shoe has yet to drop.

Psychiatrist Scott Alexander really likes and highly recommends the book, though he worries that it is not weird enough, and he thinks I overstate my case on prior futurist accuracy. Alexander assigns low moral value to the scenario I describe, even though he sees it as full of happy complex creatures. He fears it will get even worse, leading to ems who are only ever focused on their particular work task, with no mind-wandering, breaks from work, or socializing.

There are also five reviews at other blogs. (There are also three reviews at Goodreads, and one more at Amazon, which don’t mention author expertise or offer field-specific criticisms.)

Futurist and computational neuroscientist Anders Sandberg calls the book a “very rich synthesis of many ideas with a high density of fascinating arguments,” but warns “most readers will disagree with large parts of it” and “many elements presented as uncontroversial will be highly controversial.” He himself only complains that I put in “too little effort bolstering the plausibility” of the basic idea of an emulation, a topic to which he has devoted in much effort.

Education reformer Neerav Kingsland calls the book “worth reading” though he would have rather I had written more fiction. He questions our ability to foresee the results of changes this big, and he questions my prediction of low wages: “Perhaps it would become taboo to replicate yourself, akin to teenage pregnancy?”

Private investor Peter McCluskey calls the book “quite valuable” though he notes my key assumptions could end up being wrong. He wishes I would have estimated wages relative to suspense more precisely, though he felt I was borderline overconfident overall, and thought I devoted too much attention to topics like swearing, relative to topics like democracy.

Economist Peter St Onge says “The pacing is fast, chock-full of interesting ideas to play with .. Hanson has done a fantastic job.” But he sees me as “too pessimistic” because the cost to run an em is very low compared to the cost to maintain a human today, and he just can’t see marginal product of human-like labor falling that low, no matter how many workers there are.

Physicist Richard Jones, in contrast, to the above nine reviewers, criticizes just about everything but my physics. He has long criticized Eric Drexler’s efforts to apply principles of mechanical engineering to tiny chemical systems. On Age of Em, he says:

Mind uploading .. will not be possible any time soon .. The brain .. is not the product of design, it is the product of evolution, and for this reason we can’t expect there to be such a digital abstraction layer. .. It would need to incorporate a molecularly accurate model of brain development and plasticity. .. His argument is that our understanding of human nature and the operations of human societies .. is now sufficiently robust that .. meaningful predictions can be made about the character of the resulting post-human societies. I don’t find this enormously convincing. .. Hanson often is simply unable to make firm predictions; this is commendably even-handed, but somewhat undermines his broader argument. .. How do we know what forager values actually were? Very few forager societies survived in any form into historical times, .. and what we know about their values is mediated by the biases of the anthropologists and ethnographers that recorded them.

So according to Jones, we can’t trust anthropologists to describe foragers they’ve met, we can’t trust economics when tech changes society, and familiar design principles fail for understanding brains and tiny chemical systems. Apparently only his field, physics, can be trusted well outside current experience. In reply, I say I’d rather rely on experts in each field, relative to his generic skepticism. Brain scientists see familiar design principles as applying to brains, even when designed by evolution, economists see economics as applying to past and distant societies with different tech, and anthropologists think they can understand cultures they visit.

Regarding O’Connor concerns on old-fashioned mate preferences I cited a literature on that, and regarding Alexander’s zero-leisure fears the book cites a literature on max productivity breaks and vacations. Regarding Kingsland and St Onge hopes for high wages, I’ll note that though most of history before the industrial era, taboos against having kids didn’t prevent marginal productivity from typically being very low.

So far I’d say that reviews give readers reasons to suspect my emphasis is at times off, but not strong reasons to fear that Age of Em is so wrong as to be not worth reading. But more reviews are yet to come.

GD Star Rating
loading...
Tagged as: ,

Missing Engagement

On the surface, there seems to have been a big debate over the last few years on how fast automation will displace jobs over the next decade or so. Some have claimed very rapid displacement, much faster than we’ve seen in recent decades (or centuries). Others have been skeptical (like me here, here, here, and here).

On October 13, David Mindell, Professor at MIT of both Aeronautics and Astronautics, and also History of Engineering and Manufacturing weighed in on this debate, publishing Our Robots, Ourselves: Robotics and the Myths of Autonomy:

If robotics in extreme environments are any guide, Mindell says, self-driving cars should not be fully self-driving. That idea, he notes, is belied by decades of examples involving spacecraft, underwater exploration, air travel, and more. In each of those spheres, fully automated vehicles have frequently been promised, yet the most state-of-the-art products still have a driver or pilot somewhere in the network. This is one reason Mindell thinks cars are not on the road to complete automation. ..

“There’s an idea that progress in robotics leads to full autonomy. That may be a valuable idea to guide research … but when automated and autonomous systems get into the real world, that’s not the direction they head. We need to rethink the notion of progress, not as progress toward full autonomy, but as progress toward trusted, transparent, reliable, safe autonomy that is fully interactive: The car does what I want it to do, and only when I want it to do it.” (more)

In his book, Mindell expertly supports his position with a detailed review of the history of automation in planes, spacecraft and submarines. You might think than Mindell’s prestige, expertise, and detailed book on past automation rates and patterns would earn him a place in this debate on future rates of automation progress. Many of those who blurbed the book clearly think so:

“Mindell’s ingenious and profoundly original book will enlighten those who prophesy that robots will soon make us redundant.”—David Autor

“My thanks to the author for bringing scholarship and sanity to a debate which has run off into a magic la-la land in the popular press.”—Rodney Brooks

But looking over dozens of reviews Mindell’s book in the 75 days since it was published, I find no thoughtful response from the other side! None. No one who expects rapid automation progress has bothered to even outline why they find Mindell’s arguments unpersuasive.

Perhaps this shows that people on the other side know Mindell’s arguments to be solid, making any response unpersuasive, and so they’d rather ignore him. Maybe they just don’t think the past is any guide to the future, at least in automation, making Mindell’s discussion of the past irrelevant to the debate. I’ve known people who think this way.

But perhaps a more plausible interpretation is that on subjects like this in our intellectual world, usually there just is no “debate”; there are just different sides who separately market their points of view. Just as in ordinary marketing, where firms usually pitch their products without mentioning competing products, intellectuals marketing of points of view also usually ignore competing points of view. Instead of pointing out contrary arguments and rebutting them, intellectual usually prefer to ignore contrary arguments.

This seems a sad state of affairs with respect to intellectual progress. But of course such progress is a public good, where individual contributions must trade a personal cost against a collective benefit, encouraging each of us to free-ride on the efforts of others. We might create intellectual institutions that better encourage more engagement with and response to contrary arguments, but unless these are global institutions others may prefer to free-ride and not contribute to local institutions.

You might think that academic norms of discourse are such global institutions encouraging engagement. And academics do give much lip service to that idea. But in fact it is mostly empty talk; academics don’t actually encourage much engagement and response beyond the narrow scope of prestigious folks in the same academic discipline.

GD Star Rating
loading...
Tagged as: , , ,

Why Have Opinions?

I just surprised some people here at a conference by saying that I don’t have opinions on abortion or gun control. I have little use for such opinions, and so haven’t bothered to form them. Since that attitude seems to be unusual among my intellectual peers, let me explain myself.

I see four main kinds of reasons to have opinions on subjects:

  • Decisions – Sometimes I need to make concrete decisions where the best choice depends on particular key facts or values. In such cases I am forced to have opinions on those subjects, in order to make good decisions. I may well just adopt, without much reflection, the opinions of some standard expert source. I have to make a lot of decisions and don’t have much time to reflect. But even so, I must have an opinion. And my incentives here tend to be toward having true opinions.
  • Socializing – A wide range of topics come up when talking informally with others, and people tend to like you to express opinions on at least some substantial subset of those topics. They typically aren’t very happy if you explain that you just adopted the opinion of some standard expert source without reflection, and so we are encouraged to “think for ourselves” to generate such opinions. Here my incentives are to have opinions that others find interesting or loyal, which is less strongly (but not zero) correlated with truth.
  • Research – As a professional intellectual, I specialize in particular topics. On those topics I generate opinions together with detailed supporting justifications for those opinions. I am evaluated on the originality, persuasiveness, and impressiveness of these opinions and justifications. These incentives are somewhat more strongly, but still only somewhat, correlated with truth.
  • Exploration – I’m not sure what future topics to research, and so continually explore a space of related topics which seem like they might have the potential to become promising research areas for me. Part of that process of exploration involves generating tentative opinions and justifications. Here it is even less important that these opinions be true than they help reveal interesting, neglected, areas especially well-suited to my particular skills and styles.

Most topics that are appropriate for research have little in the way of personal decision impact. So intellectuals focus more on research reasons for such topics. Most intellectuals also socialize a lot, so they also generate opinions for social reasons. Alas most intellectuals generate these different types of opinions in very different ways. You can almost hear their mind gears shift when they switch from being careful on research topics to being sloppy on social topics. Most academics have a pretty narrow speciality area, which they know isn’t going to change much, so they do relatively little exploration that isn’t close to their specialty area.

Research opinions are my best contribution to the world, and so are where I should focus my altruistic efforts. (They also give my best chance for fame and glory.) So I try to put less weight on socializing reasons for my opinions, and more weight on the exploration reasons. As long as I see little prospect of my research going anywhere near the abortion or gun control topics, I won’t explore there much. Topics diagnostic of left vs. right ideological positions seem especially unlikely to be places where I could add something useful to what everyone else is saying. But I do explore a wide range of topics that seem plausibly related to areas in which I have specialized, or might specialize. I have specialized in far more different areas than have most academics. And I try to keep myself honest by looking for plausible decisions I might make related to all these topics, though that tends to be hard. If we had more prediction markets this could get much easier, but alas we do not.

Of course if you care less about research, and more about socializing, your priorities could easily differ from mine.

GD Star Rating
loading...
Tagged as: , ,

Take Origins Seriously

We have a strong tendency to believe what we were taught to believe. This is a serious problem when we were taught different things. How can we rationally have much confidence in the beliefs we were taught, if we know that others were taught to believe other things? In order to overcome this bias, we either need to find a way to later question our initial teachings so well that we eliminate this correlation between our beliefs and our early teachings, or we need to find strong arguments for why one should expect more accurate beliefs to come from the source of our personal teaching, arguments that should persuade people regardless of their teaching. These are both hard standards to meet.

We also have strong tendencies to acquire tastes. Many of the things we like we didn’t like initially, but came to like after a time. In foods, kids don’t initially like spice or bitterness, or meat, especially raw. Kids don’t initially like jogging or structured exercise, or cold showers, or fist fights, but many claim later to love such things. People find they love the kinds of music they grew up with more than other kinds. People who grow up with arranged marriages generally like them, while those who don’t are horrified. Many kids find the very idea of sex repellent, but later come to love it. Particular sex practices seem repellent or not depending on how one is exposed to them.

Now some change in tastes over time could be due to new expressions of hormones at different ages, and some can be the honest discovery of a long-term compatibility between one’s genetic nature and particular practices. But honestly, these just aren’t very plausible explanations for most of our acquired tastes. Instead, it seems that we are designed to acquire tastes according to which things seem high status, make us look good, are endorsed by our community, etc.

Now one doesn’t need to doubt culturally-acquired tastes in the same way one should doubt culturally-acquired beliefs. Once you’d gone through the early acquiring process your tastes may really be genuine, in the sense of really making you happy when satisfied. But you do have to wonder if you could come to acquire new tastes. And even if you are too old for that, you have to wonder what kind of tastes new kids could acquire. There seem to be huge gains from choosing the kinds of tastes to have new kids acquire. If they’d be just as happy with such tastes later, why not get kids to acquire tastes for hard work, for well paid work, or for products that are easier to make. For example, why not encourage a taste for common products, instead of for massive product variety?

The points I’m making are old, and often go under the label “cultural relativity.” This is sometimes summarized as saying that nothing is true or good, except relative to a culture. Which is of course just wrong. But that doesn’t mean there aren’t huge important issues here. The strong ability of cultures to influence our beliefs and tastes does force us to question our beliefs and tastes. But on the flip side, this strong effect offers the promise of big gains in both belief accuracy and happiness efficiency, if only we can think through this culture stuff well.

GD Star Rating
loading...
Tagged as: ,

Disciplines As Contrarian Correlators

I’m often interested in subjects that fall between disciplines, or more accurately that intersect multiple disciplines. I’ve noticed that it tends to be harder to persuade people of claims in these areas, even when one is similarly conservative in basing arguments on standard accepted claims from relevant fields.

One explanation is that people realize that they can’t gain as much prestige from thinking about claims outside their main discipline, so they just don’t bother to think much about such claims. Instead they default to rejecting claims if they see any reason whatsoever to doubt them.

Another explanation is that people in field X more often accept the standard claims from field X than they accept the standard claims from any other field Y. And the further away in disciplinary space is Y, or the further down in the academic status hierarchy is Y, the less likely they are to accept a standard Y claim. So an argument based on claims from both X and Y is less likely to be accepted by X folks than a claim based only on claims from X.

A third explanation is that people in field X tend to learn and believe a newspaper version of field Y that differs from the expert version of field Y. So X folks tend to reject claims that are based on expert versions of Y claims, since they instead believe the differing newspaper versions. Thus a claim based on expert versions of both X and Y claims will be rejected by both X and Y folks.

These explanations all have a place. But a fourth explanation just occurred to me. Imagine that smart people who are interested in many topics tend to be contrarian. If they hear a standard claim of any sort, perhaps 1/8 to 1/3 of the time they will think of a reason why that claim might not be true, and decide to disagree with this standard claim.

So far, this contrarianism is a barrier to getting people to accept any claims based on more than a handful of other claims. If you present an argument based on five claims, and your audience tends to randomly reject more than one fifth of claims, then most of your audience will reject your claim. But let’s add one more element: correlations within disciplines.

Assume that the process of educating someone to become a member of discipline X tends to induce a correlation in contrarian tendencies. Instead of independently accepting or rejecting the claims that they hear, they see claims in their discipline X as coming in packages to be accepted or rejected together. Some of them reject those packages and leave X for other places. But the ones who haven’t rejected them accept them as packages, and so are open to arguments that depend on many parts of those packages.

If people who learn area X accept X claims as packages, but evaluate Y claims individually, then they will be less willing to accept claims based on many Y claims. To a lesser extent, they also reject claims based on some Y claims and some X claims.

Note that none of these explanations suggest that these claims are actually false more often; they are just rejected more.

GD Star Rating
loading...
Tagged as: ,

Show Outside Critics

Worried that you might be wrong? That you might be wrong because you are biased? You might think that your best response is to study different kinds of biases, so that you can try to correct your own biases. And yes, that can help sometimes. But overall, I don’t think it helps much. The vast depths of your mind are quite capable of tricking you into thinking you are overcoming biases, when you are doing no such thing.

A more robust solution is to seek motivated and capable critics. Real humans who have incentives to find and explain flaws in your analysis. They can more reliably find your biases, and force you to hear about them. This is of course an ancient idea. The Vatican has long had “devil’s advocates”, and many other organizations regularly assign critics to evaluate presented arguments. For example, academic conferences often assign “discussants” tasked with finding flaws in talks, and journals assign referees to criticize submitted papers.

Since this idea is so ancient, you might think that the people who talk the most about trying to overcoming bias would apply this principle far more often than do others. But from what I’ve seen, you’d be wrong.

Oh, almost everyone circulates drafts among close associates for friendly criticism. But that criticism is mostly directed toward avoiding looking bad when they present to a wider audience. Which isn’t at all the same as making sure they are right. That is, friendly local criticism isn’t usually directed at trying to show a wider audience flaws in your arguments. If your audience won’t notice a flaw, your friendly local critics have little incentive to point it out.

If your audience cared about flaws in your arguments, they’d prefer to hear you in a context where they can expect to hear motivated capable outside critics point out flaws. Not your close associates or friends, or people from shared institutions via which you could punish them for overly effective criticism. Then when the flaws your audience hears about are weak, they can have more confidence that your arguments are strong.

And if even if your audience only cared about the appearance of caring about flaws in your argument, they’d still want to hear you matched with apparently motivated capable critics. Or at least have their associates hear that such matching happens. Critics would likely be less motivated and capable in this case, but at least there’d be a fig leaf that looked like good outside critics matched with your presented arguments.

So when you see people presenting arguments without even a fig leaf of the appearance of outside critics being matched with presented arguments, you can reasonably conclude that this audience doesn’t really care much about appearing to care about hidden flaws in your argument. And if you are the one presenting arguments, and if you didn’t try to ensure available critics, then others can reasonably conclude that you don’t care much about persuading your audience that your argument lacks hidden flaws.

Now often this criticism approach is often muddled by the question of which kinds of critics are in fact motivated and capable. So often “critics” are used who don’t have in fact have much relevant expertise, or who have incentives that are opaque to the audience. And prediction markets can be seen as a robust solution to this problem. Every bet is an interaction between two sides who each implicitly criticize the other. Both are clearly motivated to be accurate, and have clear incentives to only participate if they are capable. Of course prediction market critics typically don’t give as much detail to explain the flaws they see. But they do make clear that they see a flaw.

GD Star Rating
loading...
Tagged as: , , ,

Me At NIPS Workshop

Tomorrow I’ll present on prediction markets and disagreement, in Montreal at the NIPS Workshop on Transactional Machine Learning and E-Commerce. A video will be available later.

GD Star Rating
loading...
Tagged as: , ,