Comments on: Just Enough Expertise?

By: TGGP

TGGP — Fri, 18 Jan 2008 04:31:58 +0000

I said minimally informed people, which means, people who read the new york times.
If you read Caplan’s review, he points out that the “laymen” in Tetlock’s study were students who would have been far better informed than the general public.

But their choices are often no better than flipping a coin in domains with larger uncertainties.
Caplan also states in his review that Tetlock “loaded the dice”, as they say, in favor of the randomly-guessing “chimps” and against experts.

By: Robin Hanson

Robin Hanson — Fri, 18 Jan 2008 01:23:55 +0000

Let me put the challenge this way. Either there are indicators of who is more accurate or there are not. If there are no indicators, then everyone is equally likely to be accurate, and a simple average of everyone’s views would be the best we could do. If there are indicators of who is more accurate, then you should defer to the opinions of people who score better according to those indicators, in addition to feeling that those who score worse should defer to you.

By: Richard Hollerith

Richard Hollerith — Fri, 18 Jan 2008 01:16:50 +0000

minimally informed people, which means, people who read the new york times If I might offer an alternative perspective: some say the Times is better at indoctrinating than informing.

By: Jor

Jor — Thu, 17 Jan 2008 21:35:24 +0000

TGGP, let me also add, I think Caplan is right when he says experts can help frame questions properly, give choices, etc.. But their choices are often no better than flipping a coin in domains with larger uncertainties.

By: Jor

Jor — Thu, 17 Jan 2008 20:22:51 +0000

TGGP: I said minimally informed people, which means, people who read the new york times. I think if you also reference the recent discussion on this blog and Gelman’s blog about social science — you’ll realize Caplan’s larger point is wrong. Social Science experts often have a tendency to forecast well outside their knowledge, which makes them look foolish, and as Tetlock points out, no more accurate than a new york times reader. On forecasts people actually care about (i.e. the media actually asks experts about), experts often aren’t very good. If anything, I think this Tetlock’s point is one of the main reasons I favor Robin’s project, decision markets, than anything else.

By: Nick Tarleton

Nick Tarleton — Thu, 17 Jan 2008 17:28:52 +0000

if you took a short list of standard candidates for “smartest person ever” based on IQ or genius accomplishments you would do worse in terms of finding accurate sets of beliefs than if you took, say, an average elite university natural scientist.

I’m sure some (most?) of this is due to selection from different time periods. If you took the same candidates and compared them to a randomly selected elite academic from the last 2500 years, or alternatively compared the modern scientist to some candidate smartest people of the 20th century, the “smartest” groups would look a lot better, and the “smartest based on accomplishments” group might even win. Also, given the context of this discussion, you might want to exclude self-evidently crazy people like Godel, since you would exclude them if looking for the most generally reliable people.

By: Silas

Silas — Thu, 17 Jan 2008 15:03:20 +0000

komponisto: Perhaps Sabbagh thinks that an expert is just like a version of himself with a few more bits of data at hand; whereas in fact experts have much more than that: they have familiarity with long chains of reasoning that lead far away from our common everyday starting-points.

Maybe *some* experts. I seriously doubt that most experts can explain all the logical steps in reasoning, starting from layman’s knowledge, that get to their current expertise (which, btw, is my criterion for establishing whether you understand something). In my admittedly limited experience, getting an expert to explain the context of their knowledge, and on what it is based, is like pulling teeth. They seem to operate in a sort of “Chinese room”, manipulating symbols to the satisfaction of others in the field, but not understanding them.

Eliezer_Yudkowsky made a post a while ago explaining how it’s just not practical for an expert to explain to a layman all those steps. And I agree. But they should be *capable* of doing it, and that’s certainly the ideal, but I’ve just never seen it.

By: michael vassar

michael vassar — Thu, 17 Jan 2008 14:33:49 +0000

I don’t think that any theories even claim to know how g loaded accuracy of general beliefs is, but I think I made it clear from my posts that we don’t know how to measure g at the very top in any event. It’s obvious in a practical sense that, as I stated, Ed Witten is smarter than Maralyn Vos Savant regardless of what his IQ might be, and if you took a short list of standard candidates for “smartest person ever” based on IQ or genius accomplishments you would do worse in terms of finding accurate sets of beliefs than if you took, say, an average elite university natural scientist. Just look at the bios of the Mega society and the like, or of a list of important mathematicians (Newton, Godel…)

By: TGGP

TGGP — Thu, 17 Jan 2008 11:46:40 +0000

Jor, laymen are not in fact just as good as experts as far as Tetlock shows. Bryan Caplan explained that in his review, which you can access through here. Michael Vassar, do you believe the psychometric theories of a single general intelligence g are wrong or only that our metrics for measuring it fall short?

By: michael vassar

michael vassar — Thu, 17 Jan 2008 11:25:18 +0000

And Robin, I have been pointing out, in response to Rolf’s point, that we don’t have any measures of domain general extreme “intelligence” that come even remotely close to corresponding to domain general extreme accuracy. As for experience and detailed knowledge, their relevance is also always still a judgment call.