Why Broken Evals?

This review article published 36 years ago shows that it was well known back then that teacher evaluations by college students are predictably influenced by time of day, class size, course level, course electively, and more. Thus one could get more reliable teacher evaluations by building a statistical model to predict student evaluations using these features plus who taught what, and then using each teacher coefficient as that teacher’s evaluation. Yet colleges almost never do this. Why?

Actually, most orgs also use known-to-be broken worker evaluation systems:

There is a lot of systematic evidence on the connections between job performance and career outcomes. … The data shows that performance doesn’t matter that much for what happens to most people in most organizations. That includes the effect of your accomplishments on those ubiquitous performance evaluations and even on your job tenure and promotion prospects. …

[For example,] supervisors who were actively involved in hiring people whom they favored rated those subordinates more highly on performance appraisals than they did those employees they inherited or the ones they did not initially support. In fact, whether or not the supervisor had been actively engaged in the selection process had an effect on people’s performance evaluations even when objective measures of job performance were statistically controlled. (more)

So why don’t firms correct employee evaluations for this who-hired-you bias? And it isn’t just this one bias; there are lots:

Extensive research on promotions in organizations, with advancement measured either by changes in position, increases in salary, or both, also reveals the modest contribution of job performance in accounting for the variation in what happens to people. In 1980, economists … observed that salaries in companies were more strongly related to age and organizational tenure than they were to job performance. Ensuing research has confirmed and extended their findings, both in the United States and elsewhere. … One meta-analysis of chief executive compensation found that firm size accounted for more than 40 percent of the variation in pay while performance accounted for less than 5 percent. (more)

An obvious explanation here is that coalition politics dominates worker evaluations. Coalitions like being able to ignore job performance to favor their allies and punish their rivals. Winning coalitions tend to be benefiting from the current broken rules. But, you might ask, why don’t people at the top put a stop to this? Doesn’t allowing politics such free reign hurt overall org performance? This story hints at an answer:

A few years ago, Bob, the CEO of a private, venture-backed human capital software company, invited me to serve on the board of directors as the company began a transition to a new product platform and sought to increase its growth rate and profitability. Not long after I joined the board, in the midst of an upgrading in management talent, the CEO hired a new chief financial officer, Chris. Chris was an ambitious, hardworking, articulate individual who had big plans for the company— and himself. Chris asked Bob to make him chief operating officer. Bob agreed. Chris asked to join the board of directors. Bob agreed. I could see what was coming next, so I called Bob and said, “Chris is after your job.” Bob’s reply was that he was only interested in what was best for the company, would not stoop to playing politics, and thought that the board had seen his level of competence and integrity and would do the right thing. You can guess how this story ended— Bob’s gone, Chris is the CEO. What was interesting was the conference call in which the board discussed the moves. Although there was much agreement that Chris’s behavior had been inappropriate and harmful to the company, there was little support for Bob. If he was not going to put up a fight, no one was going to pick up the cudgel on his behalf. (more)

People at the top play coalition politics as hard as anyone. Rules to limit politics at lower levels can hurt lower level allies of top people, and can set expectations that limit politics at higher levels. When mob bosses who are best at violence rise to the top of a competition for boss-hood, why should they and their allies favor non-violent criteria for how to pick bosses?

Some more data:

A study using data from Dutch aircraft manufacturer Fokker reported that white-collar workers who received performance ratings of “very good” were only 12 percent more likely to be promoted than colleagues rated “good.” Meanwhile, many studies have documented the influence of numerous factors, ranging from educational credentials to race and gender, on careers, with performance often having a statistically significant but substantively small effect on advancement. For instance, a study of more than 200 employees from a variety of companies found that managers considered job tenure, educational credentials, overtime work, and absence as well as job performance in determining internal mobility for employees. A study of federal civil service employees, an excellent setting because of the extensive measures captured in the database, noted that performance ratings were weakly tied to actual productivity and that people with more educational credentials were more likely to be promoted even if they weren’t the best employees. (more)