Organisations are increasingly turning to ‘people analytics’ – using vast amounts of data about their employees – to gain insights into their workforce and introduce evidence-based decision-making. Companies hope to use measures such as clicks, views, interactions and many other online operations to accurately capture employee performance on top, or even instead of, traditional performance evaluation systems that are perceived as subjective or biased. However, our research into the use of big data analytics for performance measurement suggests that these efforts may easily backfire.

Employees become aware of data collection efforts and react by engaging in gaming strategies to influence data captured about them to their advantage. Smart organisations can respond to such reactivity of data collection in two ways. They can try to design measures that cannot be gamed or embrace unavoidable gaming.

The promise of people analytics

Evaluating and measuring employee performance in organisations is traditionally a difficult and contentious process. Performance measurement is wrought with subjectivity, lack of precision and frustration with often cumbersome corporate systems. The emerging field of people analytics, that is, the collection and analysis of vast amounts of data generated by employees using online systems, is often touted as the silver bullet. For example, Google is said to base its HR management on a complex people analytics system to uncover insights, solve problems and make decisions through analytical approaches instead of emotions, instincts and anecdotes.

The data-driven approach is understandably tempting for many organisations – it promises objectivity, fairness and significant efficiencies vis-à-vis traditional means of measuring employee performance. As a result, many companies may rush to implement increasingly scrupulous performance measurement systems, often based on clicks in online environments and applications, with the hope of capturing true worker performance. For instance, Amazon has patented wristbands that track warehouse employees’ hand movement and use haptic notifications to nudge them in certain directions to measure and increase order fulfilment rates.

But does such close monitoring through data really work?

Gaming the analytics

To find out whether people analytics stands up to its promise, we conducted research in the context of higher education where teaching and administrative activities of academics and professional services staff were constantly logged real-time in a learning analytics system. Based on the theory of reactivity by sociologists Wendy Espeland and Michael Sauder, we wanted to know if and how people change their behaviours when they know they are being observed through data. This would not only impact their actions but also limit the usefulness of such monitoring in the first place.

What we found was that employees introduced various changes to their behaviour as a result of implementing analytics. One of the most prominent and at the same time spurious effects was gaming: manipulating numbers and data to improve the appearance of performance without necessarily making the actual work practices better. For example, in order to look busy and productive, users in the system could be clicking on different content items without actually studying them. Some revealed that instead of reading required documents, scrolling through them quickly would register as completing a given activity, so that their non-compliance would not be picked up by managers. Others observed that in order to boost performance data, colleagues were posting very short but frequent forum updates (one of the performance metrics in use) as the length or intellectual depth of the content was not measured by the system.

It is clear that such gaming is detrimental to the goals of people analytics – it can partly undermine the very objectivity, fairness and efficiency of measurement that this new form of performance measurement promises. Instead of focusing on the actual work, people analytics can motivate employees to engage in sometimes elaborate schemes on how to make their work look better in analytics. This can lead to biased data and, consequently, poor decisions. With improvement being the ultimate goal of performance evaluation, there is a risk that the new analytics systems not only do little to improve performance but may even lower it as employees become distracted by playing the game of analytics.

The more data organisations collect about their employees, the more reactive the employees will become. This will only intensify the likelihood of gaming

What can organisations do?

In principle, organisations hoping to employ people analytics can set out to design analytics systems and metrics that cannot be gamed by their employees. Sometimes, there may be a way of observing and evaluating performance that will not trigger reactivity, and consequently will not lead to gaming. However, our findings suggest that this will often be difficult, if not impossible, given the fact that people are reflexive: employees constantly adjust their behaviours in response to how they perceive their own circumstances and what is beneficial to them individually.

Alternatively, organisations could try to embrace the reactivity of people analytics and use it to their advantage. Acknowledging and accepting the existence of gaming may by itself reduce the incentive to game the system. More importantly, harnessing gaming for example through gamification could also be used to improve performance, if the individual gaming behaviour could be used to drive collectively desirable outcomes. Finally, gaming performance measures is not a new phenomenon, but it will bear increasing influence on organisations as data-driven and algorithmic forms of managing work become prevalent. If employees game analytics anyway, organisations may ultimately be better off trying to turn this gaming to their advantage.

 ♣♣♣

• This blog post appeared first on the LSE Department of Management blog. It is based on the authors’ paper Closing the Loop of Big Data Analytics: the Case of Learning Analytics, Research Paper 82, ECIS 2018 proceedings, European Conference on Information Systems, Association for Information Systems.
• The post gives the views of its author(s), not the position of LSE Business Review or the London School of Economics and Political Science.
• Featured image by uglowp, under a Pixabay licence
• When you leave a comment, you’re agreeing to our Comment Policy

Marta Stelmaszak Rosa is an LSE fellow in management (information systems) at LSE’s department of management.

Aleksi Aaltonen is an assistant professor at the Richard J. Fox School of Business and Management at Temple University.

Print Friendly