LSE - Small Logo
LSE - Small Logo

Blog Admin

October 9th, 2015

Web analytics in the workplace: What Amazon and web newsrooms have in common – and where they differ.

1 comment | 1 shares

Estimated reading time: 5 minutes

Blog Admin

October 9th, 2015

Web analytics in the workplace: What Amazon and web newsrooms have in common – and where they differ.

1 comment | 1 shares

Estimated reading time: 5 minutes

Angèle_Photo-pro1-216x300The Politics of Data series continues with an investigation of how data-driven performance measurements operate in the workplace. What can we learn from the case of clicks in online news? Drawing from her ethnographic research shadowing web journalists, Angèle Christin illuminates the ambiguous and rather vague sets of meanings associated with clicks and web metrics that have become so omnipresent in online journalism.

On August 15, 2015, The New York Times published a front-page story by Jodi Kantor and David Streitfeld entitled “Inside Amazon: Wrestling Big Ideas in a Bruising Workplace.” The article examined the “data-driven” culture of Amazon, the constant competition between managers to get the highest scores, and the “purposeful darwinism” and elimination of the weakest employees that, according to the journalists, make Amazon a “bruising” workplace.

The article was an instant hit. It remained on top of the “most viewed” and “most emailed” lists for days. As of September 1, 2015, it had attracted more than 5,800 comments, the highest number ever received by an article published on The New York Times website. It was the subject of much commentary on Twitter, Reddit, and other online platforms.

While many readers concurred with the reporting featured in the article and explained in their comments that they were immediately going to cancel their subscription to Amazon Prime, others disagreed. Nick Ciubotariu, a manager at Amazon, published a post on LinkedIn – viewed more 1.2 million times – in which he criticized The New York Times article for being “blatantly incorrect” and based on anecdotal and misleading evidence: “I actually work here, and can give you a data-driven perspective of what life at Amazon is really like, today. I’m not an anonymous source, and I’m not something a journalist made up to generate clicks.” In an internal memo to all Amazon employees, Jeff Bezos recommended reading Ciubotariu’s piece in conjunction with The New York Times article. A heated discussion ensued, at The New York Times and elsewhere, about whether the journalists’ reporting was biased.

This debate struck me as particularly salient given my own research on metrics and performance measurements in online journalism. The New York Times criticized Amazon’s data-driven culture during a moment when most newsrooms are themselves becoming increasingly data-driven. As others commentators and I have written elsewhere, the popularity of online articles (clicks, unique visitors, likes, tweets, etc.) has become an important yardstick of performance and value in most web newsrooms (though The New York Times appears to remain an exception to the rise of metrics). Perhaps not surprisingly, a news story challenging data-driven workplaces fared extremely well in terms of clicks, the favored performance metric of online journalism.

This piece gives a brief analysis of how and why clicks have become such a central actor in web newsrooms before concluding with reflections on broader questions about workplace surveillance, performance measurements, and the similarities and differences between Amazon and web newsrooms.

Traffic, traffic, traffic

Over the past two decades, journalism has changed dramatically. Whereas the circulation and advertising revenues of printed newspapers continue to decline, the number of online readers, online advertising revenues, and news websites has increased exponentially, along with other forms of web publications such as blogs and social media sites (Facebook, Twitter, etc.).

Yet there is still no clear business model for the journalism of the future. A handful of prestigious publications with well-known brand names created “paywalls,” or online subscription systems, to require payment from their readers: this is the case of The New York Times, The Financial Times, and The Wall Street Journal. Most other news sites (including publications such as The Guardian, or The Huffington Post) still rely exclusively on advertising revenue, which is based on the number of monthly readers coming to the website. In other words, web traffic has become a central locus of attention for web editors struggling to maintain their advertising revenues in an extremely competitive environment.

amazon webnewsroomsImage credit: Amazon España (CC BY) and The New York Times newsroom (Public Domain)

As a consequence, a rapidly growing market has emerged for “web analytics,” that is, software programs crunching massive amounts of server data about online traffic. Google Analytics, Chartbeat, Parse.ly, Visual Revenue, Social Flow, and many others programs now track the number of “clicks,” “unique visitors,” “likes,” “shares,” “tweets,” “time engaged,” and “virality” of every bit of content published online. Most of these programs provide information in real time, actualizing results every couple of seconds. Analytics software programs have become central actors in web newsrooms. Based on this data, web editors and journalists know which articles are “flops” and which pieces are “doing well” shortly after each article is posted.

I spent several years “shadowing” web journalists and web editors in Paris and New York throughout their daily activities, listening to their comments about traffic, following their exchanges on Twitter, and then interviewing them repeatedly to find out about their careers and their interests. I also analyzed emails and archival material about the creation and development of online organizations in the United States and France and made a statistical analysis of the determinants of online success of different kinds of articles.

Based on this ethnographic research, I found that web metrics play a paradoxical role in most web newsrooms. Specifically, web metrics take at least three distinct sets of meanings depending on the context and the people who are using them.

Clicks between advertising revenues and the public sphere

First, for online news organizations, web analytics are seen as “objective” tools providing detailed information about the audience. How long do the readers stay on each page? How many readers access articles from social media sites such as Facebook and Twitter? How often do they “share” and “tweet” about articles on social networks? Whereas marketing department use this information in order to sell and negotiate the price of banners ads on the website, editors try to tweak the production of online content in order to maximize traffic. They hope to identify “niches” in their audiences, or specific groups of readers who care about a given topic and need to be catered to on a regular basis. They also rely on real-time analytics as a tool to decide which stories to “promote” or “downgrade” on the homepage based on how well they are doing. According to editors, the use of web analytics helps rationalize the relationship to the audience, at a time when traffic maximization has become an imperative in online news.

Second, web analytics are used by some editors as performance indicators for managing their personnel, especially when deciding how to promote and compensate journalists. Most analytics programs provide summary information about each author: how productive is the writer? How many visitors do his or her articles attract on average? How popular are her articles on social media? Web editors acknowledge taking into account this information when deciding which writers to hire or promote. High traffic numbers do not necessarily mean that a given journalist is talented, but, according to editors, talented journalists tend to have high traffic numbers, which then improve their career prospects. At several sites in New York and Paris, this correlation between revenue and traffic is even more clear: writers are “paid by the click,” as a percentage of the advertising revenues that their articles attract. They might also receive substantial bonuses when their articles are highly shared on social media, or when their pieces feature videos viewed by a high number of readers.

Third, web analytics are an ambivalent signal of reputation and popularity for web journalists. Journalists are well aware that “sex, scandals, and celebrities” are highly popular online and often make cynical comments on the preferences of online readers. Yet the same journalists also acknowledge that they tend to experience strong emotions, positive and negative, when their articles “go viral,” or conversely when their pieces are “flops.” Journalists explain that they write their articles with the hope that they will reach a large audience. When such an ambition fails, they feel discouraged. Clicks are more than a simple reflection of market pressures. For web journalists, online success can also be a signal of fame and involvement in the public sphere.

Algorithmic quantification turned against itself

Going back to the broader question of performance measurements, workplace surveillance, and power relations in tech companies, what can we learn from the case of clicks in online news? I conclude with two ideas.

First, we need to understand that it is because of the ambiguous and rather vague sets of meanings associated with clicks that web metrics have become so omnipresent in online journalism: everybody can project his or her own interests on them. A similar point can be made about Amazon, where white- and blue-collar workers alike tend to project many different types of meanings unto the metrics used to evaluate them.

Second, an intriguing Foucauldian pattern of contagious discipline emerges: although web metrics were originally designed to measure the behavior of others (here the audience), these numbers end up being used as a measurement of the journalists’ own professional performance. In other words, in the case of journalism, the algorithmic quantification of others was turned against the journalists themselves. Once again, a similar argument could be made about Amazon, which first began by measuring the behavior of its customers in great detail before applying the same regime to its employees.

Yet the similarities between Amazon and web newsrooms end there. What “performance” means in retail companies is fairly easy to measure, whether in terms of new clients, more items shipped, or more efficient websites. For most editors and journalists, the most significant – and least disputable – ways to assess success are not quantitative measurements, but the receipt of a Pulitzer Prize. The continued existence of a non-quantifiable measure of success remains a crucial difference between journalists on deadline and harassed tech company managers.

This is part of a wider series on the Politics of Data. For more on this topic, also see Mark Carrigan’s Philosophy of Data Science interview series and the Discover Society special issue on the Politics of Data (Science).

Note: This article gives the views of the authors, and not the position of the Impact of Social Science blog, nor of the London School of Economics. Please review our Comments Policy if you have any concerns on posting a comment below.

About the Author

Angèle Christin is a Postdoctoral Fellow at the Data & Society Institute, the Fondation de la Maison des Sciences de l’Homme, and The New School for Social Research. She completed her PhD in Sociology at Princeton University and the EHESS (Paris) in July 2014.

Print Friendly, PDF & Email

About the author

Blog Admin

Posted In: Big data | Data science | Social Media | The Politics of Data Series

1 Comments