From experience, you know that some supervisors will motivate you to do a good job while others can have a negative impact on your performance; some supervisors are supportive while others are more critical. These differences across supervisors extend to how generous supervisors are when rating the performance of subordinates during annual performance reviews. They are not innocuous as they can determine your career outcomes. Our recent research shows that being assigned to a “high-rater” for just one year will increase lifetime earnings at the firm equivalent to 6%-12% of annual earnings.

Are supervisors really different in how they rate employees? Using exceptionally rich data from the performance management system of a Scandinavian service sector firm, we uncover substantial heterogeneity in ratings across supervisors: We estimate that an employee receives on average a 30% boost in ratings when assigned to a higher-rating supervisor, which could easily move a person from a 3 (“meets expectations”) to a 4 (“exceeds expectations”). This boost can have a sizeable impact on earnings.

Where do these differences in supervisor ratings come from? A growing literature shows that supervisors do have substantial impact on the productivity of their employees. Thus it could be that some supervisors are high raters because they inspire and motivate their subordinates to higher performance. It is also well established that supervisor ratings are inherently subjective. Hence, a lenient supervisor may be more generous and reward employees with higher performance ratings, even absent better performance. These two mechanisms – managerial ability and leniency – can both potentially explain heterogeneity in supervisor ratings. They also have different empirical implications that can be used to differentiate them. In our paper, we conduct two empirical tests to establish the magnitude of heterogeneity in supervisor ratings and distinguish between the two explanations.

First, when establishing the magnitude of supervisor heterogeneity in performance ratings, we leverage the fact that supervisors and employees are observed across several years in our data. Thus we observe many employees that are rated by multiple supervisors over the years, and we observe supervisors that rate a large and changing group of employees. We can therefore establish an employee’s “typical” performance across supervisors (a measure of the employees “quality”), and we can establish how a given supervisor systematically rates employees high or low conditional on their “quality”. This leads to the estimate that high- and low-rating supervisors differ in their ratings by as much as 30%.

Second, to understand if this difference in ratings behaviour is driven by differences in supervisor’s managerial talent or their leniency, we exploit objective measures of performance (financial and other objective key performance indicators) available at the level of the teams that supervisors manage. If high-rating supervisors elicit high-performance from workers in their teams, which shows in objective team performance, their ratings most likely reflect performance rather than leniency. If high-rating supervisors, on the other hand, do not influence team performance, then the variation in ratings is more likely to be explained by the fact that some managers are lenient with their ratings and others are not. We find that high-rating supervisors tend to supervise teams that perform well. This makes it plausible that high-rating supervisors are also good managers.

Everybody benefits from good managers. When working for a good manager, workers exert more effort and in return they receive higher earnings. This combination of effort and earnings appear attractive to workers as they are more satisfied with their immediate supervisor, and they are less likely to change supervisor or quit the firm. Adding to this is that high-raters earn more themselves indicating that they are valued by the firm; a natural consequence of the fact that performance is higher in teams managed by high-rating supervisors.

Performance management systems are a ubiquitous feature of the modern workplace. Yet, such systems are often criticised for the subjectivity built into supervisor ratings. Our results show, however, that a significant part of performance ratings can be attributed to worker quality and manager induced performance. Hence, it would be premature to dismiss usage of performance ratings as they do carry important information about performance and quality in our setting.

What may be unsatisfying to companies is their imperfect insights into the nature of supervisor bias. If they were better informed about the differences in ratings styles across supervisors they could undo supervisor biases when making decisions on promotion, dismissal, bonus and pay raises for workers. While we find that the firm learns about supervisor biases with supervisor tenure, they are not eliminated. As a result, companies sometimes try to alleviate this issue by asking supervisors to grade employees on a curve. Our results suggest that this practice can be counterproductive because the grading habits by supervisors plausibly reflect productivity differences; a forced curve would constrain supervisors in their ability to motivate their teams. Hence, it might be better to train supervisors in the ratings process, enabling them to provide fair and accurate performance reviews, while still allowing them flexibility in how they use ratings.

♣♣♣

Notes:

• This blog post is based on “Supervisors and Performance Management Systems“, Journal of Political Economy, Volume 128, Number 6.
• The post expresses the views of its author(s), not the position of LSE Business Review or the London School of Economics.
• Featured image by Nadir sYzYgY on Unsplash
• When you leave a comment, you’re agreeing to our Comment Policy

Anders Frederiksen is professor in business economics and econometrics at Aarhus University, Denmark. He has been the head of the department of business development and technology since 2015, and is the director of the Center for Corporate Performance (domiciled at Copenhagen Business School). His main research area is personnel economics and more broadly labour economics. Frederiksen is research fellow at IZA-Bonn. He received his Ph.D. in economics from Aarhus University in 2005.

Lisa B. Kahn is a professor of economics at the University of Rochester. Her research focuses on labour economics with interests in organisations and education. She was previously an associate professor of economics at Yale School of Management. From 2010 to 2011 Kahn served on President Obama’s Council of Economic Advisers as the senior economist for labour and education policy. She has also been a visiting fellow at Brookings Institution and is currently a faculty research fellow at NBER and an IZA research fellow. She holds an A.B. in economics from the University of Chicago (2003) and an M.A. and a Ph.D. in economics from Harvard University (2008).

Fabian Lange is the Canada research chair in labour and personnel economics at McGill University, a research associate in the NBER’s Labor Studies Program, and a co-editor at the Journal of Labor Economics. He is the recipient of the John Rae Prize of the Canadian Economic Association for the most distinguished research record of a Canada-based economist over the period 2011-2016. His research interests concern how careers are shaped by processes of information revelation. In particular, he focuses on the role of performance management systems in modern corporations and on employer learning. He received the H. G. Lewis prize 2008 and the IZA Young Labor Economist Award for his work on employer learning. He holds a BSc in economics from the London School of Economics (1998) and a Ph.D. in Economics from the University of Chicago (2004).

Print Friendly