Mental health diagnoses are evaluated in part by the consistency with which professionals assign diagnoses. Turns out, there is often a low correlation between the diagnoses different folks assign to a patient:
The fact that psychological experts do disagree openly seems to me to be a feature, not a bug. Questioning this agreement as something homo hypocriticus could cheat around feels like a non-productive way to go here. There are plenty of cognitive biases that pshrinks actually do have, why flog the one they don't in a short post about them?
Real experts do agree. Posers andfake experts may agree among themselves, but they do not agree withthe true experts. For many of them, the kappa is less thanzero, that is the correlation with actual experts and actual realityis worse than random, that is there is anti-correlation.
Real experts know the limits of theirexpertise and very rarely make the kinds of extreme mistakes thatfake experts and posers make. Real experts can explain the basis fortheir decisions. Real expertise does not rely on gut feelings.
Some good examples of fake experts thatare (unfortunately) in charge of things is the “economists” whoare telling the politicians that tax cuts for the wealthy andausterity for the poor are exactly what is needed to stimulate theeconomy. That cutting the deficit is more important than takingadvantage of historic low interest rates to repair the infrastructurewhich needs repair and which will never be cheaper to repair thanright now while there is plenty of spare capacity to do it.
The fake experts who are AGW deniersare another. The fake experts who are evolution deniers. The fakeexperts who are HIV deniers. All of the CAM practitioners who claimto heal people with homeopathy, reiki, purging of toxins, chelation.
Is there a knob that one could turn to adjust the difficulty of coordinating?
For example, suppose that there is a prisoners-dilemma style reward structure for pointing out that a group of experts is coordinating on hair color or whatever.Then you could tune the knob until the experts are barely coordinated, and use their borderline, residual success at overcoming the difficulty of coordinating as evidence of some underlying reality to their claims.
"expert agreement remains one of the main ways the public uses to judge who is an expert."Yes, but the opposite of this is also true: we use experts' reputation to give legitimacy to decisions that are actually just subject to high randomness.
Diagnostic consistency is simply not noticed by laymen. Certainly this test won't be noticed. If common diagnoses like depression aren't consistent, laymen might notice and might care, but I don't think this is a good test of the practical consequences of DSM5. For more rare conditions, the new examiner will usually know the old diagnosis and will certainly know if the patient wants a new diagnosis.
Yes, one reason the DSM exists is to establish agreement on what to say when asked by laymen, but that doesn't mean it has an impact on diagnosis. Probably part of the problem with the recent test is that the psychiatrists aren't really aware of how far their practice is from DSM4, so they don't agree on how much a change means they should shift.
The Last Psychiatrist has a great discussion of what borderline often means. So, yes, they do manage to coordinate the way Robin suggests. But I'm not sure that they're doing it in order to look like experts.
Borderline: ...1. Very attractive female, who comes for problems the psychiatrist considers ordinary: men, work/school, problems with parents, etc. It is diagnosed here most often by female psychiatrists, and carries the connotation: "Grow up."...
I'm not sure that humans can be *that* hypocritical.
Note that experts do agree, and recognize experts, and to some extent signal status, by having a common vocabulary. Some jargon is undeniably useful, but it's hard to argue that many fields don't use more than strictly necessary.
Does this hold true for economists? My perception is that the public, generally, often refuses to accept expert economic consensus (in, for instance, favoring rent control laws).
Maybe another condition is that people will only recognize experts in fields where they don't have a strong viewpoint themselves?
psychiatry sucks but it's hella betta than having folks fire speedballs
The fact that psychological experts do disagree openly seems to me to be a feature, not a bug. Questioning this agreement as something homo hypocriticus could cheat around feels like a non-productive way to go here. There are plenty of cognitive biases that pshrinks actually do have, why flog the one they don't in a short post about them?
Real experts do agree. Posers andfake experts may agree among themselves, but they do not agree withthe true experts. For many of them, the kappa is less thanzero, that is the correlation with actual experts and actual realityis worse than random, that is there is anti-correlation.
Real experts know the limits of theirexpertise and very rarely make the kinds of extreme mistakes thatfake experts and posers make. Real experts can explain the basis fortheir decisions. Real expertise does not rely on gut feelings.
Some good examples of fake experts thatare (unfortunately) in charge of things is the “economists” whoare telling the politicians that tax cuts for the wealthy andausterity for the poor are exactly what is needed to stimulate theeconomy. That cutting the deficit is more important than takingadvantage of historic low interest rates to repair the infrastructurewhich needs repair and which will never be cheaper to repair thanright now while there is plenty of spare capacity to do it.
The fake experts who are AGW deniersare another. The fake experts who are evolution deniers. The fakeexperts who are HIV deniers. All of the CAM practitioners who claimto heal people with homeopathy, reiki, purging of toxins, chelation.
Is there a knob that one could turn to adjust the difficulty of coordinating?
For example, suppose that there is a prisoners-dilemma style reward structure for pointing out that a group of experts is coordinating on hair color or whatever.Then you could tune the knob until the experts are barely coordinated, and use their borderline, residual success at overcoming the difficulty of coordinating as evidence of some underlying reality to their claims.
"expert agreement remains one of the main ways the public uses to judge who is an expert."Yes, but the opposite of this is also true: we use experts' reputation to give legitimacy to decisions that are actually just subject to high randomness.
Diagnostic consistency is simply not noticed by laymen. Certainly this test won't be noticed. If common diagnoses like depression aren't consistent, laymen might notice and might care, but I don't think this is a good test of the practical consequences of DSM5. For more rare conditions, the new examiner will usually know the old diagnosis and will certainly know if the patient wants a new diagnosis.
Yes, one reason the DSM exists is to establish agreement on what to say when asked by laymen, but that doesn't mean it has an impact on diagnosis. Probably part of the problem with the recent test is that the psychiatrists aren't really aware of how far their practice is from DSM4, so they don't agree on how much a change means they should shift.
The Last Psychiatrist has a great discussion of what borderline often means. So, yes, they do manage to coordinate the way Robin suggests. But I'm not sure that they're doing it in order to look like experts.
Borderline: ...1. Very attractive female, who comes for problems the psychiatrist considers ordinary: men, work/school, problems with parents, etc. It is diagnosed here most often by female psychiatrists, and carries the connotation: "Grow up."...
I'm not sure that humans can be *that* hypocritical.
Note that experts do agree, and recognize experts, and to some extent signal status, by having a common vocabulary. Some jargon is undeniably useful, but it's hard to argue that many fields don't use more than strictly necessary.
Does this hold true for economists? My perception is that the public, generally, often refuses to accept expert economic consensus (in, for instance, favoring rent control laws).
Maybe another condition is that people will only recognize experts in fields where they don't have a strong viewpoint themselves?