Expert At Versus Expert On

A prosperous and successful plumber is an expert at plumbing.   Someone who is a good source for accurate information on plumbing is an expert on plumbing.  More generally, an expert at a topic is someone who has gained the most attention, praise, income, and so on via their association with the topic.   But this may not be the best expert on that topic.  He may have succeeded by not giving the most accurate information, but by telling people what they want or expect to hear, or by entertaining them.

We often rely on the heuristic of looking to an expert at a topic, when what we want is an expert on a topic.  In fact, most of the people we see being labeled as "experts" are primarily experts at topics.  For example, TV talking heads discussing topic X are usually people who have made a successful career in X.  We may see a general talk about war, or a CEO talk about business.   

But it a rare field where the best way to succeed is to always be completely honest with everyone about everything.   We could greatly benefit from better ways to determine who is really an expert on a topic.  Prediction markets are one possibility.

GD Star Rating
loading...
Tagged as:
Trackback URL:
  • Matthew

    You have two definitions of expertise in this post. I disagree slightly with the first one. To me, being an expert at a particular task implies a high level of competency in actually doing the task. Being an expert on a particular subject implies a high degree of knowledge about the subject. Often, these two sides of expertise are found in the same person, but if I had a burst pipe in my basement (and I had to choose between the two categories), I’d prefer an expert at plumbing. If I was producing a TV show about plumbing I’d probably prefer an expert on plumbing.

    I don’t think that those people we might agree to call experts on a subject are above manipulating the truth a bit to make their favorite theories look better.

    I think that which type of expert you want depends on why you want them, and that their honesty and reliability have to be evaluated on a case by case basis.

  • Stuart Armstrong

    “Those who can’t, teach.” And those who can’t be experts at a subject, become experts on a subject. At least, that’s what people think, especially if the money imbalance is high – the CEO paid more than the economist, the plumber paid more than the expert on plumbing.

    The experts at somthing also have the advantage that they’ve demonstrated some level of competence at it, shown some actual knowledge – while “experts” on a subject may be completely unreliable.

    But if prediction markets did work here, it would solve both issues – get us some reliable info on the experts, and make them richer – this would reassure people that didn’t become experts on just because they couldn’t be experts at.

    Robin, as an expert on prediction markets 🙂 can you say whether they’re likely to work well in this case?

  • Stuart Armstrong

    “In theory, theory and practice are the same; in practice, they’re not.”
    But the distinctions between the experts on and experts at does diminishs when you go to the more theoretical subjects – experts on physics are normally those who’ve made a career in physics.

    The gulf between experts on and experts at for a particular field may tell us about the level of (dis-)honesty needed to succeed in that field.

  • http://profile.typekey.com/robinhanson/ Robin Hanson

    Matthew, if someone claimed you needed to replace your pipes so that they would not burst, what kind of expert would you want to evaluate that claim?

  • eddie

    More generally, an expert at a topic is someone who has gained the most attention, praise, income, and so on via their association with the topic.

    The definition Robin provides of an “expert at” versus “expert on” seems a poor one – or at least not one fitting with my understanding of the terms.

    “Expert at” implies expertise gained by activity. Furthermore, one can only be an expert “at” some activity, not any subject in general. There are no “experts at animals”, although there are experts at animal husbandry, veterinary medicine, and zookeeping. “Expert at” implies proficiency at certain skills, and by further implication the posession of certain knowledge related to those skills.

    “Expert on” implies expertise gained by study. It can apply to any subject that can be studied – while you can’t be an “expert at animals” you can certainly be an “expert on animals”. It implies the posession of certain knowledge but not the posession of any particular skills, except perhaps the skill of scholarship.

    The choice of whether to seek advice from an “expert at” or an “expert on” depends on whether you want to best accomplish something or best learn about something – whether you would benefit most from advanced skills or advanced knowledge. Even considering that, if I were looking only for knowledge, I would still be leery of overvaluing the academic and undervaluing the professional. The successful professional at least has the check of reality to validate that at least some aspect of his world view may be correct. The successful academic can be well studied, have devoted a long time to his field, have lots of knowledge and understanding, be successful in his career, and still be dead wrong.

  • eddie

    He may have succeeded by not giving the most accurate information, but by telling people what they want or expect to hear, or by entertaining them.

    A plumber is unlikely to succeed by those means (maybe a celebrity plumber? plumber to the stars?). A general or CEO might, to the degree that their position is due to political skill rather than business or military acumen. How many “experts at” can become successful without being proficient in their supposed area of expertise?

    But it a rare field where the best way to succeed is to always be completely honest with everyone about everything.

    I don’t understand this statement. What does honesty have to do with either expertise at or expertise on?

  • albatross

    It seems to me that the distinction you’re making here is:

    a. Expert at X = someone successful in X, for good or bad reasons

    b. Expert on X = someone who is a real expert about X

    I guess it almost feels like you’re distinguishing between real experts and convincing fakes with good credentials. I’ll acknowledge that distinguishing these from outside the field is very hard. The best solution I know of is to look for evaluations from other experts. I may not be able to distinguish a good cardiologist from a bad one, but maybe my normal doctor can.

    I think a more useful distinction might be between someone who knows how to do X, and someone who is good at explaining X to others. For example, we all know people who are good at something, but are rotten teachers or aren’t good at explaining their work to anyone else. And I can think of people who are not first-tier experts in any area of their field, but who are fantastic at explaining their field to outsiders.

    More broadly, you want people who can integrate their expertise about X into a wider base of knowledge. If I want to know whether human-caused global warming is a likely impending crisis, I don’t just want someone who knows all there is to know about climate modeling, I want someone who understands the whole field, including background assumptions and history.

  • http://jed.jive.com Jed Harris

    Two points:

    First: It seems that Robin is an expert on expertise by his own definition (I’m not even that). So by Robin’s definition, who today would be an expert at expertise? Maybe someone who has to pick experts based on their ability to actually perform tasks, and who suffers consequences if their chosen experts don’t get the job done?

    Second: We have “markets” in “expertise at”. The most transparent I know is open source development groups. While there is still some of the usual “expertise at” (i.e. success via rhetoric), the “audience” in the bigger open source projects is pretty good at noticing whether you write code, and whether your code works well and solves problems. So these communities may be very useful to study in understanding how to manage expertise.

    As a follow on to that point: The non-ownership and non-transactional policies in open source avoid many of the intrinsic problems with information goods transactions. Issues such as unresolvable asymmetric information between “buyer” and “seller” don’t arise because there’s no need to keep some information secret until the transaction is consummated.

    So maybe the relative transparency of “expertise at” in open source is not a coincidence.

  • Agent00yak

    “Experts on” will be biased to give intelligent sounding explanations in order to preserve their status, which is more at risk than “experts at”. An “expert at” is much more likely to tell you the truth when the truth is simple. A good example of this phenomenon would be economists (experts on markets) vs traders (experts at markets) explaning market movements. Unless the trader has a large position in the effected market (and hence likely to lie), the trader is the one to ask about an anomalous market situation.

  • http://jed.jive.com Jed Harris

    Sorry, I meant “While there is still some of the usual ‘expertise on’…”. I hope it was obvious.

    In my experience the most “prosperous and successful plumber[s]” are not actually the best experts at plumbing — the experts at plumbing tend to do OK but the experts at promoting themselves make more money. (I’m speaking here about my unfortunately extensive experience with a wide variety of actual plumbers working in my house. So I guess I qualify as experienced, if not an expert, at finding good plumbers.)

    The best way to find actual experts at plumbing seems to be voluntary reputation data. Word of mouth is too limited. I have had good results with a local reputation web site and a (non-profit) reputation “buyer’s guide” based on surveys. The comparable for-profit services aren’t all that useful. Surely it is significant that this niche is filled by non-profits that aggregate voluntary reviews, but I don’t know what that fact signifies.

  • http://profile.typekey.com/mpianalto/ Matthew Pianalto

    This distinction makes me think of chicken sexers: there aren’t many of them left (thanks to genetics which make sexing chicks much easier), but in the olden days, there were these folks who could determine the sex of baby chicks with over 99% accuracy at rates of hundreds of chicks per hour on the job for 12 hour stretches, even though they could not tell an inquirer HOW they knew the difference between males and females. (The point of sexing the chicks was basically to get rid of the males to save money on feed.) These people were experts at chicken sexing, and could train others to do it, too, but the whole process seemed mysterious. Researchers (don’t recall the reference) discovered that they do use perceptual cues, albeit “unconsciously.” (I believe this is the sort of “unconscious perception” that Malcolm Gladwell writes about in Blink.

  • TGGP

    Stuart Armstrong’s first post is part of the reason I pay little heed to philosophers/ethicists. Has anyone ever had a pressing need to hire a philosopher? Maybe to teach philosophy so the students may later use their knowledge to…teach philosophy.

  • http://profile.typekey.com/sentience/ Eliezer Yudkowsky

    Usually I explain my distaste for e.g. academic “bioethics” by saying “ethicists write mainly for an audience of other ethicists”. Thanks to this post, I can now put it more clearly: Practicing engineers who care about people tend to become experts at the ethics of their profession, but ethicists who publish papers for other ethicists merely become expert at sounding wise. And what the Singularity Institute would like to see is experts at AI ethics, who will probably not go by the job title of “AI ethicist”.

  • B.S.

    TGGP,

    Nobody receives a direct private benefit from philosophers. Philosophers provide more of a public good. When an idea is good, the positive externalities are immeasurable. When an idea is wrong, the negative externalities are equally immeasurable.

  • TGGP

    Yann Martel also seems to think he provides a vital public good. There is no cost to claiming that to be the case, an incentive to claim it regardless of whether or not it is true and no reliable method I’ve heard of for determining who is actually providing a “public good” in this manner (for all I know someone’s output might be a negative externality on us) and to what extent.

  • Matthew

    Stuart,

    “Matthew, if someone claimed you needed to replace your pipes so that they would not burst, what kind of expert would you want to evaluate that claim?”

    OK, sticking to the example of plumbing.

    It would depend on the context. If I got a call from a telemarketer offering me a great deal on replumbing my house, I would ignore it. If a friend of mine, who happens to be a plumber, warns me that my original plumbing was ineptly installed and it really should be redone, I’d be inclined to trust him. If I’m having minor plumbing problems (low water pressure, leaky faucet, etc) and the plumber I contact tells me that it’s going to be a very expensive fix, I would probably get a second opinion. If available, I might look for reviews (there are online and offline venues for this), or I might ask for recommendations from friends or family.

    “The gulf between experts on and experts at for a particular field may tell us about the level of (dis-)honesty needed to succeed in that field.”

    I’m not sure what you mean by this statement, could you clarify please?

  • Stuart Armstrong

    “The gulf between experts on and experts at for a particular field may tell us about the level of (dis-)honesty needed to succeed in that field.”

    I’m not sure what you mean by this statement, could you clarify please?

    Yes! Robin said that experts “at” would be those successful in the field. If the experts “on” are not very successful (which would be the case if the “at” and “on” were very different) that means that there is a set of additional skills that experts “at” have, beyond knowledge of the field.

    The most obvious skills to explain the gulf are people skills, or the ability to lie/market well. If we have a profession where a good lie helps the expert “at” (say, plumbing or real estate) and if the gulf between them and experts “on” is wide, then we can suspect that lying plays a big role in that separation.

    Other skills that might explain the gulf – ambition, drive, lack of social snobbery, intuition – are not quite so convincing, as there should be many experts “on” who could acquire them easily, if the incentive is there. But lying is a skill you will not acquire unless you practice it, so is the best candidate if the gulf is large.

    PS: I did not write the “pipes” comment, but your response seems quite close to Robin’s initial point: those who are good “at” a subject are not the ones you want to call, unless you can use incentives to get them closer to experts “on”.

  • Matthew

    Stuart,

    Sorry about the misattribution, I looked at the name above instead of the name below the comment.

    I think I now have a better idea of what Robin means by “expert at” and “expert on”. The distinction seems to be between those that are successful and those that are skilled. That wasn’t (and isn’t) my intuitive interpretation of his terms, but the original post makes more sense to me in that light.

    I took exception to the terms expert at and expert on perhaps because it reminded me of the opinion some people have in my field (physics) that theorists are somehow more “pure” than experimentalists.

  • albatross

    So, is there a good generic way to distinguish between experts at and on? Most externally-available measures (length of publication list, credentials, position, honors received) are meaningful, but are also subject to being gamed–we probably all know people in our fields who publish a lot of papers relative to their actual contribution in the field, and people who publish few papers, but with a lot of value per paper. Similarly, credentials (PhD from NameDroppingUniversity) tell you something, but maybe not as much as you’d like to know. And position might be a matter of good connections or networking skills or even nepotism.

    Sometimes, evaluations from other experts who’re familiar with their work are the best solution. Other times, there’s some objective measure that can be used, though probably any measure will have a certain susceptibility to be gamed.

    It seems like prediction markets could help here if there were some people who knew enough to do a good job with the evaluation, and if they knew who they were. (That is, if you’re the assistant of the cardiologist with a steller reputation and a graveyard full of mistakes nobody knows about, you could make lots of money predicting future failure.) But this clearly is a hard kind of thing to evaluate, even for relative insiders–look at how tenure decisions sometimes get made!

  • Michael Sullivan

    I don’t think lying/marketing is the best explanation for a gap between experts at and experts on in most situations, including plumbers. I know something about plumbing, just because I’m mechanical and interested, though I would not consider myself anywhere near an expert on it. But my expertise *at* it is far smaller still, because I while I understand the principles of how plumbing works, I’ve never attempted any more than minor fixes. I’m also large and have fairly average finger dexterity and arm strength for a person my size, and probably less patience than average. In attempting to do a few plumbing jobs myself, I’ve discovered that while I knew what to do, actually getting various pieces apart without breaking them was big fat PITA. I mostly don’t do that anymore. My point is not that I know more than a random professional plumber (I don’t), merely that it is obvious to me that many of the skills necessary for them to succeed are incidental to the topic, and to decision such as whether your pipes need to be completely replaced. Someone who is physically disabled could well be the best person to make the latter decision, but they won’t be the best person to actually replace your pipes.

    Let’s take another subject I know even more about. I’m an owner of a print shop. I know a lot about the capabilities of presses, and about the chemistry and mechanics involved in making them work. I can often make correct decisions about the effect of an unusual adjustment that skilled operators cannot. In some ways I’m more of an expert *on* presses than they are. But if I tried to actually run the press tomorrow, I’d be lucky to produce 20% of the work that my best operators do. They not only have some physical skills that are superior to mine, they have practice at doing the work every single day, and have developed a rhythm. Lying or marketing is not a significant part of their success (except to the extent that they might get me to believe that some PITA job is actually impossible), but there’s still a pretty big gulf between their expertise at, and my expertise on.

    Coaching sports would be another good example. Plenty of the best coaches never played their sport at the highest levels, because they simply did not have the physical gifts required.

    In any case I don’t think this is purely a distinction of successful vs. skilled, although that may be one dimension. I think it’s also about two different skills, one which involves immediate performance and is easier to measure, and one which involves pure knowledge and is harder to measure.

  • sourcreamus

    If the people who picked sports commentators understood this difference the world would be a much better place.

  • Cameron Taylor

    I suggest a third alternative.

    – Expert on: Knows all about it.
    – Expert at: Can do it really well.
    – Expert in: Has a lot of success while doing it.

  • Kiki

    I don't agree that experts at are the ones usually consulted. It varies from field to field. It is the lack of balance that is the problem.

    For example in the fields Innovation and Entrepreneurship, experts on predominate. And this limits effective policy making.

    We need a balance of advice from both experts at and experts on – both have their merits.

  • http://shagbark.livejournal.com Phil Goetz

    Expert at baseball: Plays well.
    Expert on baseball: Knows lots of statistics.

    Expert at religion: A theologian.
    Expert on religion: An atheist.

    Expert on animal behavior: An ethologist.
    Expert at animal behavior: A cat.

  • michael vassar

    Actually Phil, I’m pretty sure that many theologians don’t differ detectably on how their beliefs cash in from most atheists.

  • David

    This post is so full of bias I cannot help but wonder if it’s actually a “trick question” trying to elicit responses ;-).

    “A prosperous and successful plumber is an expert at plumbing. Someone who is a good source for accurate information on plumbing is an expert on plumbing.”

    They are not mutually exclusive.

    “More generally, an expert at a topic is someone who has gained the most attention, praise, income, and so on via their association with the topic. But this may not be the best expert on that topic.”

    Of course not.

    “He may have succeeded by not giving the most accurate information, but by telling people what they want or expect to hear, or by entertaining them.”

    But an expert on a topic might succeed for the same reasons? Because they are funny or good looking or…

    “We often rely on the heuristic of looking to an expert at a topic, when what we want is an expert on a topic.”

    As noted above, they are not mutually exclusive and in instances where they diverge, we might in fact want *both*.

    “In fact, most of the people we see being labeled as “experts” are primarily experts at topics. For example, TV talking heads discussing topic X are usually people who have made a successful career in X. We may see a general talk about war, or a CEO talk about business.”

    Really?! I’ve noticed just the opposite. Frequently the people labeled experts on TV are experts on the topic, not experts at the topic.

    “But it a rare field where the best way to succeed is to always be completely honest with everyone about everything.”

    Hard to refute that statement considering how absolute it is. I don’t believe I know of a living human that is completely honest with everyone about everything. In the context of this discussion I’d be concerned not only with honesty on the part of an expert, but equal with cognitive bias.

    “We could greatly benefit from better ways to determine who is really an expert on a topic. Prediction markets are one possibility.”

    Finally something we can agree on.

  • Sergio Pinski

    I am a cardiologist myself. From my perspective:

    Expert at: person who often makes and implements decisions, with good (measurable) outcomes far exceeding an occasional bad one.

    Expert on: person who knows all (or most) the published/reported data on a particular topic.

    These categories are not mutually exclusive. If you’re seeking medical help, you’ll aim at the Expert at.

  • Pingback: Reading Yudkowsky, part 20

  • Pingback: Reading Yudkowsky, part 22

  • Pingback: Expert at | Keritesorszag