More Broken Evals
Back in January I quoted:
In 1980, economists … observed that salaries in companies were more strongly related to age and organizational tenure than they were to job performance. Ensuing research has confirmed and extended their findings, both in the United States and elsewhere. … One meta-analysis of chief executive compensation found that firm size accounted for more than 40 percent of the variation in pay while performance accounted for less than 5 percent. (more)
Part of the reason may be that employee performance evaluations are often political. From an ’87 paper:
Our research approach involved in-depth, semi-structured interviews with 60 executives. … from seven large organizations and represented 11 functional areas. As a group, they averaged more than 20 years of work experience and more than 13 years of managerial experience. ..
Executives admitted that political considerations nearly always were part of the [employee] evaluation process. One vice-president summarized the view these executives shared regarding the politics of appraisal:
As a manager, I will use the review process to do what is best for my people and the division. … I’ve got a lot of leeway – call it discretion – to use the process in that manner. … I’ve used it to get my people better raises in lean years, to kick a guy in the pants if he really needed it, to pick up a guy when he was down or even to tell him that he was no longer welcome here. It is a tool that the manager should use to help him do why it takes to get the job done. I believe most of us here at —- Operate this way regarding appraisals. … Accurately describing an employee’s performance is really not as important as generating ratings that keep things cooking.
Executives suggested several reasons why politics were so pervasive and why accuracy was not their primary concern. First, executives realized that they must live with subordinates in a day-to-day relationship. Second, they were also very cognizant of the permanence of the written document. .. Perhaps the most widespread reason … was that the formal appraisal was linked to compensation, career, and advancement in the organization. …
Executives generally believed the appraisal process became more political and subjective as one moved up the organizational ladder:
The higher you rise in this organization the more weird things get with regard to how they evaluate you. … The process becomes more political and less objective and it seems like the rating process focuses on who you are as opposed to what you’ve actually accomplished … As the stakes get higher, things get more and more political. ..
Although not frequently reported, a few executives admitted to giving a higher rating to a problem employee to the get employee promoted “up and out” of the department. …
A deliberately deflated rating was sometimes used to teach a rebellious subordinate a lesson. … Deflated ratings were also used as part of a termination procedure. First, a strongly negative rating could be used to send and indirect message to a subordinate that he or she should consider quitting. …. Second, once the decision has bee made that the situation was unsalvageable, negative ratings could then be used to build a strongly documented case against the marginal or poor performer.
To underscore my point (post below), why do you say "broken" rather than "healed" or "saved?"
Employee evaluation is like science in the sense that how well it's done affects all employees through it's effect on the success of the company. It's not like science because it isn't under public scrutiny and isn't expected to be reproduced. It's also not like science in that there is no hypothesis being tested against the measurements.
Evaluation by objective criteria is like the rule of law in that it forces the employer to describe goals in a way that the employee can guide their actions toward (whether the expressed goals turn out helpful or not). We think the rule of law is good for [edit] certain general commands in [/edit] a command economy. Coase said that firms have economic reasons to be like command economies. Is it a categorical switch whether it's appropriate to act like a command economy or a market in all ways? Does the transaction-cost model mean that small companies should be more commandy and large companies should be more markety? (If so that would suggest a reason why bosses in larger companies bend the rules more.) But somehow rules for a year of an employee's behavior don't seem to me necessarily right for small companies either.
Overall, I guess companies need to *signal* objectivity and fairness, but in their actual behavior negotiate and communicate hard-to-articulate, fluctuating values and make sharp-edged, hard-to-justify decisions, i.e. act political and unfair.
But that's a guess, my question is whether politics or objectivity would be more "broken" in this context. [edit] Also, how political and politically-driven is the process of setting up the on-the-surface objective criteria and review processes? What needs would you expect the set-up process to serve and what needs does it actually serve? [/edit]
And what has been done or what would you do to find out?
Are there ways to evaluate whether "objective" (i.e., formal rule-driven) performance evaluations are a more accurate, or (separate question) less political, way to evaluate the value of an employee to an organization? Otherwise this seems a one-sided comparison. Is politics correlated with accuracy, correlated with inaccuracy, or orthogonal? Are the formal review criteria used by large organizations more or less accurate than those used in small organizations? More or less political in large organizations? Even the notion of accuracy in predicting the value of an employee seems suspect to me, since it's like looking at a single price within a market and asking whether it's "correct".
When I ask how political formal criteria are, I don't mean whether they specify objective measures. I mean whether the measures selected are selected for more or less political reasons or by more or less political processes.