Tag Archives: Disagreement

Leading Isn’t About Info

The study recruited 150 participants and divided them into groups of three. One person was randomly assigned to be the group’s leader; all were told they could contribute advice, but that the leader was responsible for making the decision. Then they undertook a group task: choosing a job candidate. Of 45 items of information about the candidate, some were given to all three, and some to only one of the participants. … Group members rated the most narcissistic leaders as most effective. But … the groups led by the greatest egotists chose the worse candidate for the job. … “The narcissistic leaders … inhibited the communication because of self-centeredness and authoritarianism.” … Good leaders facilitate communication by asking questions and summarizing the conversation—something narcissists are too self-involved to do. (more; HT Karl Mattingly)

I predict that as the above result becomes more widely known, we will not much change our tendency to choose egotists as leaders. Yes the ability to get subordinates to reveal unusual info is valuable to the organization as a whole, but today this ranks pretty low among the many competing considerations in choosing a leader, and that probably won’t change much in the foreseeable future.

GD Star Rating
loading...
Tagged as: , ,

More Disagreement Is Far

More evidence that we tend to be less truth-oriented in far mode:

[Researchers] found participants were more persuaded to change their opinion after receiving concrete, detailed messages from a spatially near rather than distant source. These results fit well with recent [Near-Far] work on distancing and persuasion, which found that persuasion was highest when participants experienced a small (as opposed to large) amount of distance and received low-level, concrete (rather than high-level, abstract) persuasive messages. …

[Researchers] found that participants were less affected and less likely to seek information about a potential unpleasant truth (e.g., missing an opportunity on the stock market) when it concerned a distant rather than near situation (i.e., foreign vs. local company). (more)

GD Star Rating
loading...
Tagged as: ,

Pigheaded Specialists

Kumar: “Excuse me, Professor Jones, could you explain why I was rejected for graduate study in your illustrious science program. I’m Vinay Kumar.”

Jones: “Ah yes, Mr. Kumar. You had excellent grades and test scores, and your teachers testified to your intelligence and work habits. However, you scored poorly on our new `scientist’ personality test – your pigheadedness score was near the bottom. We are sorry, but the NSF’s new rules for promoting progress give us no choice but to reject you.”

Kumar: “So I’m too pigheaded to be a scientist?”

Jones: “No, you are not pigheaded enough. You are too likely to objectively evaluate research on its methods, rather than on whether its conclusions match your previous positions. You are too willing to change your mind when evidence changes.”

Kumar: “But isn’t such objectivity a scientific ideal?”

Jones: “Well some say so, but the wise know otherwise. If all scientists were like you, they’d agree too much, and all work on the same projects. For robust scientific progress, we need different scientists to work on different projects. And to get that, we need them to pigheadedly draw different conclusions from the same data.”

Kumar: “But our modern economy is based on a vast specialization of labor. Are you saying that modern economies would be impossible without us all being pigheaded, because otherwise we’d all do the same job? Can’t we just pay people to do different jobs?”

Jones: “Money might motivate ordinary people to do different jobs, but in the magestarium of science money matters not. Scientists only pick research topics based on scientific beliefs. So to get different research, we need different beliefs.”

Pretty crazy, right? Behold this oped by Cordelia Fine in Saturday’s New York Times:

Scientists … rated the paper’s methodology, data presentation and scientific contribution significantly more favorably when the paper happened to offer results consistent with their own theoretical stance. … This is a worry. Doesn’t the ideal of scientific reasoning … shun the ego-driven desire to prevail over our critics … ? Perhaps not. Some academics have recently suggested that a scientist’s pigheadedness and social prejudices can peacefully coexist with — and may even facilitate — the pursuit of scientific knowledge. …

An irrational tendency like pigheadedness can be quite an asset in an argumentative context. A engages with B and proposes X. B disagrees and counters with Y. Reverse roles, repeat as desired — and what in the old days we might have mistaken for an exercise in stubbornness turns out instead to be a highly efficient “division of cognitive labor” with A specializing in the pros, B in the cons. It’s salvation of a kind: … by way of positive side effect, these heated social interactions, when they occur within a scientific community, can lead to the discovery of the truth. (more; HT Eric Schliesser)

This idea that humans disagree because disagreement is good for society, by getting people to take different actions, is an old one. I’m rather skeptical about it in general, and especially regarding academic research. Academics are easily motivated by money and other personal perks, and if such payments are insufficient, I see no advantages to a diversity of beliefs that a diversity of values can’t supply. Given enough value diversity, I see no net social gain from folks being unwilling to update based on evidence.

GD Star Rating
loading...
Tagged as: ,

Who Wants Accuracy?

A well-connected reporter (who I promised I’d keep anonymous) just told me that a major Washington media organization started a project studying major media pundits, and a big part of this project was assessing individual pundit forecast track records. After several months of several folks working on the project, it was killed, supposedly because management decided readers don’t care as much about pundit accuracy as they’d previous thought.

Of course that need not have been their real reason – perhaps some folks didn’t like the ratings it was giving to their favorite pundits. Or perhaps it died for any of a hundred random reasons projects are killed. Even so, I found this anecdote interesting.

GD Star Rating
loading...
Tagged as: , ,

Seek Criticism

Two weeks ago I read Penrose’s new book Cycles of Time. I enjoyed his review of the time’s arrow puzzle, and was intrigued by his proposal that distances fade away in vast infinite futures, allowing them to become tiny flat big bangs again. But not only did Penrose wave his arms pretty wildly on how there could be a metric along which metrics would disappear in approaching the vast-tiny border, he seems to make a very elementary mistake in positing that entropy could have a similar magnitude in our big bang post and our vast distant future, because info is lost in evaporating black holes. The entropy in black hole radiation is more than the holes themselves, which is far more than a tiny flat big bang before.

Raphael Bousso (co-author of that Anthropic breakthrough I raved about in ’08) reviews the book in Science, and seems to agree:

Penrose is at his best when he explains this deep and beautiful mystery, and the book may be worth reading for this chapter alone. However, he compounds the shortcomings of his cyclic universe model when he argues that it can solve the low-entropy problem. At this point, another idea is introduced: like vacuum cleaners, black holes appear to reduce disorder by swallowing matter. By the end of one “aeon,” Penrose argues, most matter has ended up in giant black holes. Very little entropy remains, and the next aeon can commence in perfect order. The second law guarantees that a vacuum cleaner does not actually decrease the overall disorder; at best, it just shifts it around. In fact, the machine creates far more entropy than it destroys (for example, by heating up the air in the room). A black hole, it turns out, is not different. Penrose’s assertion that black holes destroy entropy is flatly contradicted by “the generalized second law of thermodynamics”. (more)

How could such a big-shot make such a simple mistake? One should seriously consider the possibility that he isn’t saying what he appears to be saying, and in fact is saying something much more clever and insightful. But if so why wouldn’t he have devoted more effort to explaining, to avoid the misunderstanding. His book reads as if he didn’t even consider that this criticism would be offered. And that fact leads me to believe Penrose considers himself to be such a big shot that he didn’t even ask colleagues to read and criticize his book before publication. And that sort of isolation makes me more willing to believe that he did in fact just make a simple mistake.

The lesson: no matter how much better you think you are than the lowly incompetents that surround you, you’d still do well to ask for and listen to criticism.

GD Star Rating
loading...
Tagged as: ,

Against DIY Academics

Vladimir M:

In many areas, … respectable academic authors are the richest and most reliable source of information, and people claiming things completely outside the academic mainstream are almost certain to be crackpots. … [But] what happens when a field fails both of them, having no clear research directions and at the same time being highly relevant to ideologues and interest groups? Unsurprisingly, it tends to be really bad.

The clearest example of such a field is probably economics, particularly macroeconomics. … Even a casual inspection of the standards in this field shows clear symptoms of cargo-cult science: weaving complex and abstruse theories that can be made to predict everything and nothing, manipulating essentially meaningless numbers as if they were objectively measurable properties of the real world, experts with the most prestigious credentials dismissing each other as crackpots.

Now it might be that academics in fields low in ideological conflict tend to accept each other’s work too easily, to protect the reputation of their field. In this case, fields with more ideological conflict could be more reliable, if that conflict led to more critical examination of results.

However, let us accept for the sake of argument that all else equal in ideological fields intellectual progress is slower, and claims tend to be make with more overconfidence.  What exactly would this imply for your beliefs about this area?

It certainly wouldn’t imply that you ignore what experts write. Yes, it makes sense to adjust your beliefs for the average overconfidence there, but even with large adjustments your best estimates should still rely heavily on average expert estimates.  After all, even if they know less than they think, they still know a lot more than you.

I suspect that what Vladimir and others usually have in mind is Do It Yourself Science:

Looking at the data yourself and drawing your own conclusions.

Now trying your own hand at the subject can help you to understand most any subject.  It can help you discern who are the real experts, and better understand what they say.  There’s a reason students are asked to do labs and problem sets.

But if you plan to mostly ignore the experts and base your beliefs on your own analysis, you need to not only assume that ideological bias has so polluted the experts as to make them nearly worthless, but you also need to assume that you are mostly immune from such problems!

Yes, this is a natural assumption to make, as we rarely feel that we are subject to the biases we suspect we see in others.  But without substantial evidence clearly supporting it, this is mostly just wishful thinking.  If ideology severely compromises others’ analysis on this subject, then most likely it severely comprises yours as well.  You should mostly just avoid having opinions on the subject.  But if you must have reliable opinions, average expert opinions are probably still your best bet.  (Unless of course you have a prediction market available. :) )

GD Star Rating
loading...
Tagged as: ,

Beware “Consensus”?

If your doctor discourages you from seeking another opinion, you have even more reason to get one. (more)

Honest contrarians who expect reasonable outsiders to give their contrarian view more than normal credence should point to strong outside indicators that correlate enough with contrarians tending more to be right. (more)

Perhaps one strong outside indicator that a contrarian view is right is when the media goes out of its way to say that it is opposed by a “scientific consensus”! Ron Bailey in July:

Several [out of the eight media-declared] scientific consensuses before 1985 turned out to be wrong or exaggerated, e.g., saccharin, dietary fiber, fusion reactors, stratospheric ozone depletion, and even arguably acid rain and high-dose animal testing for carcinogenicity.

It seems to me that for folks with a contrarian bent, getting more better studies like this should be a high priority. More details from Ron: Continue reading "Beware “Consensus”?" »

GD Star Rating
loading...
Tagged as: ,

What Is “Belief”?

Richard Chappell has a couple of recent posts on the rationality of disagreement. As this fave topic of mine appears rarely in the blogsphere, let me not miss this opportunity to discuss it.

In response to the essential question “why exactly should I believe I am right and you are wrong,” Richard at least sometimes endorses the answer “I’m just lucky.” This puzzled me; on what basis could you conclude it is you and not the other person who has made a key mistake? But talking privately with Richard, I now understand that he focuses on what he calls “fundamental” disagreement, where all parties are confident they share the same info and have made no analysis mistakes.

In contrast, my focus is on cases where parties assume they would agree if they shared the same info and analysis steps.  These are just very different issues, I think.  Unfortunately, they appear to be more related than they are, because of a key ambiguity in what we mean by “belief.”  Many common versions of this concept do not “carve nature at the relevant joints.”  Let me explain.

Every decision we make is influenced by a mess of tangled influences that can defy easy classification. But one important distinction, I think, is between (A) influences that come most directly from inside of us, i.e., from who we are, and (B) influences that come most directly from outside of us. (Yes, of course, indirectly each influence can come from everywhere.) Among outside influences, we can also usefully distinguish between (B1) influences which we intend to track the particular outside things that we are reasoning about, from (B2) influences that come from rather unrelated sources.

For example, our attitude toward rain soon might be influenced by (A) our dark personality, that makes us expect dark things, and from (B1) seeing dark clouds, which is closely connected to the processes that make rain.  Our attitude toward rain might also be influenced by (B2) broad social pressures to make weather forecasts match the emotional mood of our associates, even when this has little relation to if there will be rain.

Differing attitudes between people on rain soon is mainly problematic regarding (B1) aspects of our mental attitudes which we intend to have track that rain. Yes of course if we are different inside, and are ok with remaining different in such ways, then it is ok for our decisions to be influenced by such differences. But such divergence is not so ok regarding the aspects of our minds that we intend to track things outside our minds.

Imagine that two minds intend for certain aspects of their mental states to track the same outside object, but then they find consistent or predictable differences between their designated mental aspects. In this case these two minds may suspect that their intentions have failed. That is, their disagreement may be evidence suggesting that for at least one of them other influences have contaminated mental aspects that person had intended would just track that outside object.

This is to me the interesting question in rationality of disagreement; how do we best help our minds to track the world outside us in the face of apparent disagreements? This is just a very different question from what sort of internal mental differences we are comfortable with having and acknowledging.

Unfortunately most discussion about “beliefs” and “opinions” are ambiguous regarding whether those who hold such things intend for them to just be mental aspects that track outside objects, or whether such things are intended to also reflect and express key internal differences. Do you want your “belief” in rain to just track the chance it will rain, or do you also want it to reflect your optimism toward life, your social independence, etc.?  Until one makes more clear what mental aspects exactly are referred to by the word “belief”, it seem very hard to answer such questions.

This ambiguity also clouds our standard formal theories. Let me explain.  In standard expected-utility decision theory, the two big influences on actions are probabilities and utilities, with probabilities coming from a min-info “prior” plus context-dependent info. Most econ models of decision making assume that all decision makers use expected utility and have the same prior. For example, agents might start with the same prior, get differing info about rain, take actions based on their differing info and values, and then change their beliefs about rain after seeing the actions of others. In such models, info and thus probability is (B1) what comes from outside agents to influence their decisions, while utility (A) comes from inside. Each probability is designed to be influenced only by the thing it is “about,” minimizing influence from (A) internal mental features or (B2) unrelated outside sources.

In philosophy, however, it is common to talk about the possibility that different people have differing priors. Also, for every set of consistent decisions one could make, there are an infinite number of different pairs of probabilities and utilities that produce those decisions. So one can actually model any situation with several expected-utility folks making decisions as either one with common priors or with uncommon priors.

Thus in contrast to the practice of most economists, philosophers’ use of “belief” (and “probability” and “prior”) confuses or mixes (A) internal and (B) external sources of our mental states. Because of this, it seems pointless for me to argue with philosophers about whether rational priors are common, or whether one can reasonably have differing “beliefs” given the same info and no analysis mistakes. We would do better to negotiate clearer language to talk about the parts of our mental states that we intend to track what our decisions are about.

Since I’m an economist, I’m comfortable with the usual econ habit of using “probability” to denote such outside influences intended to track the objects of our reasoning.  (Such usage basically defines priors to common.) But I’m willing to cede words like “probability”, “belief” or “opinion” to other purposes, if other important connotations need to be considered.

However, somewhere in our lexicon for discussing mental states we need words to refer to something like what econ models usually mean by “probabilities”, i.e., aspects of our mental states that we intend to track the objects of our reasoning, and to be minimally influenced by other aspects of our mental states.

(Of course all this can be applied to “beliefs” about our own minds, if we consider influences coming from our minds as if it were something outside.)

GD Star Rating
loading...
Tagged as: , ,

The Wisdom Of Others

When we choose to act on our own private clues, or to infer the clues of others from their actions, we too often neglect the wisdom of others:

In situations where it is empirically optimal to follow others and contradict one’s own information, the players err in the majority of cases, forgoing substantial parts of earnings. The average player contradicts her own signal only if the empirical odds ratio of the own signal being wrong, conditional on all available information, is larger than 2:1, rather than 1:1 as would be implied by rational expectations. … In total, the meta dataset contains 29,923 decisions made by 2,813 participants in 13 studies. All participants observe a private signal and a (possibly empty) string of previous choices made by others in analogous situations. In all decision problems there are two actions and two possible payoffs, but the dataset nevertheless comprises a large variety in environments, instructions, players’ personal characteristics, and histories of other players’ choices. (more)

Of course copying others’ acts sends a bad signal about our confidence in own own info and and analysis abilities. So it can make sense to focus more on one’s own clues to the extent is is important to send a positive signal to observers.  Just beware of assuming too easily that such gains are substantial.

GD Star Rating
loading...
Tagged as: ,

Tug-O-War Is Not Charity

Arnold Kling:

Ezra Klein thinks that political organizations are worthwhile charities.

If you donate money to a food bank, it can provide only as much food as your money can buy. If you donate it to a nonprofit that specializes in food policy issues, it can persuade legislators to pass a new program – or reform an existing one – that can do much more than any single food bank.

So he winds up giving his money to support a think tank whose employees are somewhere around the 95th percentile of the income distribution, in the hope that they will help tilt the rent-seeking in Washington in a direction that he likes. … It is actually sort of sad for a policy wonk to settle on the idea of making donations to an organization of policy wonks.

If public policy is a point in a high dimensional space, then every policy change has two components: a partisan and a non-partisan change. Partisan changes are along standard partisan axes, where people are lined up in a tug-o-war on different sides pulling in different directions. Non partisan changes, in contrast, are not seen as a win for one side relative to others. Technically, partisan changes project total changes into the partisan subspace.

Assuming all parties think they seek good, partisan changes can only be good if some parties are right while others are wrong about what is good. In contrast, you can be right about a non-partisan change without others being wrong. Since the total space has a far larger dimension that the partisan space, there is a huge scope for searching in that larger space for changes that all sides could see as good. And donations to encourage such efforts can indeed consistently produce large social gains relative to their costs.

Donations to change policy within the partisan subspace, however, only achieve good when they happen to be on the right side of partisan disagreements. Averaged over the disagreeing parties, such donations cannot on average achieve good unless there is a correlation between between donations, or donation effectiveness, and which sides are right.  Even if you think you are right at the moment on your particular partisan policy opinions, you can’t think it good on average to encourage partisan donations, unless you think donations tend overall to go to the good or more donation-effective sides.

Unfortunately most thinktank efforts go into pushing for their sides within the partisan subspace, because that is what most donors care about. For example, Ezra’s two concrete policy examples, of “the need for food banks and homeless shelters and social services” and “repeal the 2010 health-care reform legislation,” are both clearly partisan.

Humans clearly tend to be overconfident about politics. Since you are human, that tendency is a likely cause of your confidence in your political opinions. If your politics were about doing good with policy, you should correct for that overconfidence, and that correction would on average move folks to have little interest in partisan pushes.  Of course if your politics is not about policy, but about showing loyalty, how clever or informed you are, etc., well then go right ahead and be partisan. But don’t tell me that is generally beneficial charity.

GD Star Rating
loading...
Tagged as: , ,