Author Archives: Andrew

The pinch-hitter syndrome: a general principle?

A few years ago I was checking an article that was about to be published in a statistics journal and I noticed that the copy editor had made a bunch of stupid changes that I then had to go back and fix. This has also happened for two of my books.

This is a funny thing. A copy editor is a professional editor. All they do (or, at least, much of what they do) is edit, so how is it that they do such a bad job compared to a statistician, for whom writing is only a small part of the job description?

The answer certainly isn’t that I’m so wonderful. Non-copy-editor colleagues can go through anything I write and find lots of typos, grammatical errors, confusing passages, and flat-out mistakes. (And check out the long list of errata for the first printing of our first book!)

No, the problem comes with the copy editor, and I think it’s an example of the pinch-hitter syndrome. The pinch-hitter is the guy who sits on the bench and then comes up to bat, often in a key moment of a close game. When I was a kid, I always thought that pinch hitters must be the best sluggers in baseball, because all they do (well, almost all) is hit. But of course this isn’t the case–the best hitters play outfield, or first base, or third base, or whatever. If the pinch hitter were really good, he’d be a starter. So, Kirk Gibson in the 1988 World Series notwithstanding (I was watching that on TV–that gives me credit for being there, right?), pinch hitters are generally not the best hitters.

There must be some general social-science principle here, about generalists and specialists, roles in an organization, etc?

GD Star Rating
loading...
Tagged as:

Are brilliant scientists less likely to cheat?

In this discussion of Allegra Goodman’s book novel Intuition, Barry wrote, "brilliant people are at least as capable of being dishonest as ordinary people."  The novel is loosely based on some scientific fraud scandals from the 1980s, the one of its central characters, a lab director, is portrayed as brilliant and a master of details, but who makes a mistake by brushing aside evidence of fraud by a postdoc in her lab.  One might describe the lab director’s behavior as "soft cheating" since, given the context of the novel, she had to have been deluding herself by ignoring the clear evidence of a problem.

Anyway, the question here is:  are brilliant scientists at least as likely to cheat?  I have no systematic data on this and am not sure how how to get this information.  One approach would be to randomly sample scientists, index them by some objective measure of "brilliance" (even something like asking their colleagues to rate their brilliance on a 1-10 scale and then taking averages would probably work), then do a through audit of their work to look for fraud, and then regress Pr(fraud) on brilliance.  This would work if the prevalence of cheating were high enough.  Another approach would be to do a case-control study of cheaters and non-cheaters, but the selection issues would seem to be huge here, since you’d be only counting the cheaters who got caught.  Data might also be available within colleges on the GPA’s and SAT scores of college students who were punished for cheating; we could compare these to the scores of the general population of students.  And there might be useful survey data of students, asking questions like "do you cheat" and "what’s your SAT" or whatever.  I guess there might even be a survey of scientists, but it seems harder to imagine they’d admit to cheating.

Continue reading "Are brilliant scientists less likely to cheat?" »

GD Star Rating
loading...
Tagged as:

What is a taboo question?

From Greg Mankiw I saw this newspaper article by Steven Pinker, "In defense of dangerous ideas:  In every age, taboo questions raise our blood pressure and threaten moral panic. But we cannot be afraid to answer them."

I see two connections to "overcoming bias" here.  First, and most obviously, Pinker is suggesting that, by ignoring these "taboo questions," we as a society will suffer from the consequences of bias (not knowing whether certain policies will work well, etc.).  Below I claim the opposite, that Pinker’s very choice of topics to study is itself suffused with bias which he doesn’t seem to recognize.

The second connection is that I think that we, the "overcoming bias" crowd, tend to feel that many of our ideas are themselves taboo in some sense.  By pushing our own dangerous ideas, we may be more Pinker-like than we realize.

Here is (some of) Pinker’s article, followed by a pretty long point-by-point discussion of many of his examples, along with (what I see as) his implicit assumptions).  Pinker writes:

Do women, on average, have a different profile of aptitudes and emotions than men?

Were the events in the Bible fictitious — not just the miracles, but those involving kings and empires?

Has the state of the environment improved in the last 50 years?

Continue reading "What is a taboo question?" »

GD Star Rating
loading...
Tagged as:

Calibration in chess

Daniel Kahneman posted the following on the Judgment and Decision Making site:

Have there been studies of the calibration of expert players in judgments of chess situations — e.g., probability that white will win?

In terms of the amount and quality experience and feedback, chess players are at least as privileged as weather forecasters and racetrack bettors — but they don’t have the experience of expressing their judgments in probabilities. I [Kahneman] am guessing that the distinction between a game that is "certainly lost" and "probably lost" is one that very good players can make reliably, but I know of no evidence.

Despite knowing much less about decision making and (likely) less about chess than Kahneman, I have three conjectures:

Continue reading "Calibration in chess" »

GD Star Rating
loading...
Tagged as:

Goals and plans in decision making

For years, Dave Krantz has been telling me about his goal-based model of decision analysis.  It’s always made much more sense to me than the usual framework of decision trees and utility theory (which, I agree with Dave, is not salvaged by bandaids such as nonlinear utilities and prospect theory).  But, much as I love Dave’s theory, or proto-theory, I always get confused when I try to explain it to others (or to myself):  "it’s, uh, something about defining decisions based on goals, rather than starting with the decision options, uh, …."  So I was thrilled to find that Dave and Howard Kunreuther just published an article describing the theory.  Here’s the abstract:

We propose a constructed-choice model for general decision making. The model departs from utility theory and prospect theory in its treatment of multiple goals and it suggests several different ways in which context can affect choice.

It is particularly instructive to apply this model to protective decisions, which are often puzzling. Among other anomalies, people insure against non-catastrophic events, underinsure against catastrophic risks, and allow extraneous factors to influence insurance purchases and other protective decisions. Neither expected-utility theory nor prospect theory can explain these anomalies satisfactorily. To apply this model to the above anomalies, we consider many different insurance-related goals, organized in a taxonomy, and we consider the effects of context on goals, resources, plans and decision rules.

The paper concludes by suggesting some prescriptions for improving individual decision making with respect to protective measures.

Continue reading "Goals and plans in decision making" »

GD Star Rating
loading...
Tagged as:

How should unproven findings be publicized?

A year or so ago I heard about a couple of papers by Satoshi Kanazawa on "Engineers have more sons, nurses have more daughters" and "Beautiful parents have more daughters."  The titles surprised me, because in my acquaintance with such data, I’d seen very little evidence of sex ratios at birth varying much at all, certainly not by 26% as was claimed in one of these papers.  I looked into it and indeed it turned out that the findings could be explained as statistical artifacts–the key errors were, in one of the studies, controlling for intermediate outcomes and, in the other study, reporting only one of multiple potential hypothesis tests.  At the time, I felt that a key weakness of the research was that it did not include collaboration with statisticians, experimental psychologists, or others who are aware of these issues.

Continue reading "How should unproven findings be publicized?" »

GD Star Rating
loading...
Tagged as: , ,

Reply to “Libertarian Optimism Bias …”

In this entry on George Bernard Shaw and G. K. Chesterton, I noted that 50 or 100 or 200 years ago, leftists associated progress with material happiness while rightists were more skeptical and tended to say that progress wasn’t always such a good thing.   Nowadays, the debates usually go in the other directions, with people on the left being less positive about material progress and people on the right saying that things are great now and are getting better.

Will Wilkinson replied here, analyzing the current differences between left and right as different views on government intervention:

Nothing beats a "crisis" to rally support for a big government effort. Right statists constantly drum up moral panics about sex and drugs. Also, Mexicans are "invading" and terrorists will surely blow us all up while singing the Star Spangled Banner at baseball games if we don’t allow the executive Jack Bauer to torture military detainees whenever he wants. Similarly, left statists warn that the shores of Manhattan will be inundated by rising oceans and very cute baby polar bears will die in droves. Also, inequality is soaring, threatening the foundations of democracy. And the middle class lives in terrifying "economic insecurity." And so on.

This is an interesting point.  Once again, it might be helpful to compare with attitudes 50, 100, etc. years ago.  Shaw, like many other socialists, supported government intervention in the economy and also thought that material progress would give us higher living standards and better lives.  (To put it in Wilkinson’s framework, Shaw saw problems with society that he thought could be alleviated by government intervention, but he framed this in an optimistic view of material progress, rather than in a "there’s more to life than just money" attitude.)

GD Star Rating
loading...
Tagged as:

Bayes: radical, liberal, or conservative?

I wrote the following (with Aleks Jakulin) to introduce a special issue on Bayesian statistics of the journal Statistica Sinica (volume 17, 422-426).  I think the article might be of interest even to dabblers in Bayes, as I try to make explicit some of the political or quasi-political attitudes floating around the world of statistical methodology.

Continue reading "Bayes: radical, liberal, or conservative?" »

GD Star Rating
loading...
Tagged as:

Were people better off in the middle ages than they are now?

G. K. Chesterton writes, at the end of his celebrated book on George Bernard Shaw:

I know it is all very strange.  From the height of eight hundred years ago, or of eight hundred years hence, our age must look incredibly odd.  We call the twelfth century ascetic.  We call our own time hedonist and full of praise and pleasure.  But in the ascetic age the love of life was evident and enormous, so that it had to be restrained.  In a hedonist age pleasure has always sunk low, so that it had to be encouraged. 

How high the sea of human happiness rose in the Middle Ages, we now only know by the colossal walls that that they built to keep it in bounds.  How low human happiness sank in the twentieth century our children will only know by these extraordinary modern books, which tell people that it is a duty to be cheerful and that life is not so bad after all.  Humanity never produces optimists till it has ceased to produce happy men. 

It is strange to be obliged to impose a holiday like a fast, and to drive men to a banquet with spears.  But this shall be written of our time:  that when the spirit who denies beseiged the last citadel, blaspheming life itself, there were some, there was one especially, whose voice was heard and whose spear was never broken.

Continue reading "Were people better off in the middle ages than they are now?" »

GD Star Rating
loading...
Tagged as:

Total vs. marginal effects, or, Are the overall benefits of health care “probably minor”?

I was having an interesting discussion with Seth Roberts about his claim that "the overall benefits of health care are probably minor."

Continue reading "Total vs. marginal effects, or, Are the overall benefits of health care “probably minor”?" »

GD Star Rating
loading...
Tagged as: