The impact of technological innovation is a hot topic in the 21st century. Debates around Big Data (BD) and the technology revolution keep trending. Ignoring the scale of the social changes and organisational disruption created by the apparently exponential growth of technology and the information that has become available to individuals and organisations is impossible. Of particular interest to business is how to utilise the large amounts of data available to make better decisions and improve business performance. Technology companies have been promoting BD for over a decade, while governments, public institutions, economists and business commentators all appear keen not to be left behind in the ‘big data revolution’. The Harvard Business Review suggested that BD represented a ‘revolution in management’ while others claimed it would “transform business processes and alter corporate ecosystems”, creating “the next frontier for innovation, competition and productivity”.
In 2014 Gartner argued that BD was re-defining the relationship between human and machine. Two years later however, they suggested that the hype was over. Having gone over the peak of inflated expectations, BD had fallen into the trough of disillusionment. Whether it will emerge from the trough remains to be seen. We, and colleagues, wanted to get beyond the hype, to see what was actually going on and to consider how businesses might respond. In our paper we highlighted the lack of credible empirical research in the area and questioned whether BD has fundamentally changed business and management. We argued that the challenge for organisations is to develop strategies that exploit BD to add value to the bottom line. Unless BD can be harnessed to manage day-to-day operations, and enhance performance, what’s the point?
We suggest that just as happened during the gold rushes of the 19th century, there are fortunes to be made from Big Data. However, then, as now, few prospectors have found the rich seams of gold that they imagined. The age old problem that accompanies all the hype is knowing where the gold might be and once located having the wherewithal to extract it. This is not necessarily easy; but it seems there are fortunes to be made. Whether by selling the metaphorical claims (databases and data streams), shovels (BD software) and expertise (consultancy), there is money to be made selling to eager prospectors.
Big Data Prospecting
And so as with previous ‘hypes’, those selling “the dream” are perhaps doing rather better than the businesses desperate to improve their productivity. In 1984, the economist Robert Solow pointed out that, “You can see the computer age everywhere but in the productivity statistics.”. Has anything changed? Whilst there are always exemplars (such as Amazon and Uber) that can be used to promote the benefits of BD, finding credible evidence of improved performance is not so easy.
Arguably businesses haven’t really changed much in the 21st century – service providers still provide services, manufacturers make ‘stuff’ and retailers still sell it. Technology is in some ways making the world a tougher place to do business. Organisations have to adapt to a volatile environment where consumer-led innovation and ever-increasing process automation seems the norm. In a complex, changing operational environment, no matter how much data we collect, the data available is only ever an historical record of past events. Whatever we imagine real time data is always old data. And, the more data we have and the faster we collect it, the more complex and volatile the environment we inhabit becomes. No amount of data provides certainty of what will happen in the future. What seems clear is that data alone cannot revolutionise business. Unless you know what you want from the data, an enormous constantly changing set of data creates more uncertainty, not less. Perhaps paradoxically, we need to know what we want from data (whether big or small) to find the answer we are looking for.
Turning big data into business opportunities
We can’t seize business opportunities if we cannot turn the data we have into meaningful information. So it becomes apparent that developing the management skills to be able to work in an environment of seemingly limitless data is probably the biggest challenge we face. We need to interpret data quickly and effectively by extracting patterns that reveal what has happened and indicate what is likely to happen in the future. It is by looking for patterns and anomalies in those patterns, that we are able to make sense of what is going on. Only then have we been able to create knowledge from the data. Done well this allows for exciting and innovative decision-making. Done badly – poor decision making creates huge operational inefficiencies and costs, which all too often can go unnoticed until it is too late.
Realising digital opportunities
However assuming that BD can be analysed to generate reliable and robust information does it not guarantee customer satisfaction and improved profitability? There remain a number of people-related challenges to be considered, and overcome, if the promise of BD is to be realised. We need to think about human behaviour, how we think, how we respond to the information we have. Management narratives stress the importance of speed and responsivity. But is speed a good thing? Are quick decisions better and more informed than those that we take our time over? As the saying goes, fools rush in where angels fear to tread. Similarly we are conditioned to think that more data is better data. How much data can we cope with to optimise what we do and how we do it? What we want is the ability to exploit data, any data, big or small, new or old, so that we can set new levels of performance and new agendas for business.
This requires a re-evaluation of the skills and competences we need from the workforce. Similarly we need to reconceptualise customers. They are far more transient and at the same time they have more control of the relationships they have with businesses. They have become the co-creators of value and they expect to interact with your business using the channel of their choice, at a time of their choice. Technology allows customers to be in constant dialogue with businesses and organisations using whatever channel they choose. More data and more technology makes the customer a complex, ever changing and unique entity.
Solow’s paradox and Gartner’s acknowledgement of the BD hype cycle suggest everyone will not strike gold. We suggest that after the Gold Rush business success will increasingly depend on the ability to move from big data to small data (and even from fast data to slow data). Managers and organisations who can step back from the white noise of BD and identify patterns within it in order to make sense of what is going will be the ones that thrive in our digital world.
- This blog post is based on the author’s paper Making sense of Big Data – can it transform operations management? International Journal of Operations & Production Management 37(1):37-55
- The post gives the views of its author, not the position of LSE Business Review or the London School of Economics.
- Featured image credit: Pile of sheets, by Noj Han, under a CC-BY-SA-2.0 licence
- Before commenting, please read our Comment Policy.
Olga Matthias is a Senior Fellow at the University of Bradford’s Operations and Information Management Group. Previously, she had a long career at PA Consulting Group, in the Business Operations and Performance practice. After completing a graduate training scheme at Lloyds Bank she moved into management consultancy, where she worked for a number of companies, including Cap Gemini, before joining PA Consulting. Her consulting work focused on customer service and performance improvement through enhancing process, people skills and IT. She led large business-transformation programmes for blue-chip companies in the UK and in New Zealand, concentrating on aligning business and systems operations. Olga’s research interests are multidisciplinary in nature, and developed from the deep experience gained from her consulting clients, for whom she improved networks, relationships and organisational performance. Olga is on the Editorial Board of the Journal of Operations Management. She is Visiting Associate Professor of Operations Management at the University of Exeter Business School.
Ian Fouweather is a Lecturer in Business Operations at Bradford University. He has been a part time lecturer in a range of business disciplines for over ten years, with a broad interest in practical management issues. Ian has recently become a permanent member of staff at the faculty, but teaches for a variety of business schools around the world. Ian also runs a small business consultancy supporting client organisations to change, adapt and improve, at both a strategic and tactical level.
A very valuable and timely article on an area that desperately needs to emerge from the hype. All good organisations use data (not necessarily big data) to help make decisions. Perhaps the best result that could emerge from the Big Data enthusiasm is a better understanding of how to use data in decision making.