LSE - Small Logo
LSE - Small Logo

Michael Schrage

October 13th, 2020

Recommendation engines: data-driven self-help is the future of the internet

0 comments | 4 shares

Estimated reading time: 5 minutes

Michael Schrage

October 13th, 2020

Recommendation engines: data-driven self-help is the future of the internet

0 comments | 4 shares

Estimated reading time: 5 minutes

Self-knowledge, self-improvement. Self-control. In his 1859 best-selling ‘Self-Help,’ Samuel Smiles celebrated the self-disciplined cultivation of character and capability. “The spirit of self-help is the root of all genuine growth in the individual,” that eminent Victorian declared. Smiles’ book not only codified cultural standards for industrial age ‘self-helpers,’ it launched the genre that made Dale Carnegie, Steven Covey and Oprah Winfrey inevitable. Never underestimate the market power and value of advice.

Time to turn the page on aspirational best-sellers: Self-help’s spirit and substance has gone digital. From Coursera, Udemy and instructional YouTube videos to Silicon Valley’s ‘quantified self’ – ‘self-knowledge through numbers’- movement, increasingly intelligent technology makes increasingly fertile soil for genuine individual growth. Today’s internet is already a global paradise for autodidacts; just ask your mobile. Machine learning everywhere transforms human learning everywhere. A smarter internet means smarter auto-didacts. Data-driven ‘self-help’ is the internet future.

The most powerful and pervasive engine driving personal transformation is the recommendation engine. That social media prompt suggesting a text to a forgotten colleague; the serendipitously timed article in your feed you’d never have seen; the unexpected song energising your day; the personal – or professional – introduction you couldn’t have imagined: recommenders generate the bespoke options and choices that, with a tap or swipe, turn ‘next moments’ into graspable opportunities. They invite instant self-improvement; they make a ‘better self’ more possible. Smiles, who also pioneered engineering history and biography, would surely appreciate how recommendation engines now empower more people worldwide than steam engines.

The most innovative recommendation engines learn to cultivate self-awareness in their most faithful users: Spotify’s Discover Weekly promises music-lovers personalised playlists of songs they’ve never heard before but will surely enjoy. Netflix algorithms make binge-worthy video programming not just possible but predictable. Indeed, Alibaba’s Taobao – the world’s biggest e-commerce site – insists it knows its online shoppers even better than they know themselves. As Amazon’s Jeff Bezos, the world’s richest entrepreneur, would cheerfully agree, insightful recommendations make for great business.

But seeing recommendation engines primarily as engines of commerce misses the far larger point: these technologies are less about the ‘future of business’ than the future of the self. Recommendation engines don’t just influence consumption, they pro-actively shape the investments – social, financial, cultural, educational – people make in themselves. Smart humans already understand that it’s dumb to ignore good advice from smart machines.

By fomenting global revolutions in how ‘choice’ is personalised, presented, experienced and understood, recommenders have transformed ‘self-help’ expectations. Recommendation becomes less about ‘what you want to buy’ than ‘who you want to be.’ Truly well-engineered, highly-attuned and data-rich recommendation engines drive self-discovery. Why do you make the choices you do? What do you want to do next?

As articulated by Smiles and other champions of character and competence, ‘self-help’ is firmly rooted in everyday choice. To the extent ‘you are your choices,’ recommendation engines increasingly drive one’s decision guidance systems. The recommendations people follow – and ignore – reflect who they think (and feel) they really are and could be. Like intimate personal mirrors, recommenders magnify how individuals see themselves, their preferences and possibilities.

Bluntly, this is the AI that matters most – not the ‘artificial intelligence’ of future robots but the ‘augmented introspection’ of today’s humans. With no apologies to Socrates, algorithmic recommendation radically updates and upgrades the Delphic challenge to ‘Know Thyself.’

What the ‘artificial intelligentsia’ misunderstand or minimise is that no matter how ‘smart,’ ‘autonomous’ and/or ‘conscious’ technologies become, their biggest impact will be on mortal self-awareness. Indeed, ubiquitous recommenders inadvertently create perhaps the greatest natural experiment in ‘free will’ and agency in history. Just how open will most people be to ‘good advice’ from their machines? Will they reliably defer and comply? Or will they defiantly question and challenge? Or will other novel ‘self-help’ equilibria emerge? Those questions are no longer rhetorical.

That’s because recommendation engineers draw heavily from psychology, neuroscience and Nobel Prize winning economics research on ‘choice architectures’ to train their intelligent algorithms. They seek to identify the ‘framings,’ ‘nudges’ and ‘defaults’ people willingly embrace and which ones they wilfully defy. Those decision insights are invaluable; so is how they were determined. Machine learning’s seemingly irresistible rise virtually guarantees that future nudges, framings and defaults will become more irresistible. Yes, smarter machines assure the choices will get better – but for whom? Who will ‘choice architect’ the choice architects?

That’s why the ethics of tomorrow’s ‘self-help’ are as important as its genius technologies. Will next-generation recommendation engines be more driven to discover and exploit user insecurities or identify and reinforce customer strengths? Cui bono? Who benefits?

The better recommenders know or anticipate their users, the more important that fundamental question becomes. Enterprises reliant on digital manipulation, trickery and deceit to achieve desired outcomes have made their intentions clear. ‘Recommendation engines’ whose ‘choices’ are computationally skewed, misleading or exploitive are frauds. They represent the antithesis of Smilesian ‘self-help.

This would be as true for a government public health ‘Nudge-Unit-In-Your-Mobile’ recommenders as ones from eco-friendly carbon neutrality promoting NGOs. Recommendation engines work with integrity when they measurably empower agency and choice. Self-help and ‘augmented introspection’ become impossible when ‘choice architectures’ rest on dishonest foundations.

That’s why the recommendation future will likely become more visible, transparent and accessible – via market forces or regulatory diktat. This massive global experiment can’t help but generate provocative hypotheses into what makes ‘self-help’ work. Will better bespoke recommendations and advice lead to better people and outcomes? Do digital ‘nudges’ and ‘prompts’ help build character or breed computational co-dependence? Humanity is just beginning to seriously ask those questions. The answers will determine how individuals – and their cultures – ultimately choose to better themselves.

♣♣♣

Notes:

  • This blog post is based on the author’s book “Recommendation Engines“, MIT Press.
  • expresses the views of its author(s), not the position of LSE Business Review or the London School of Economics.
  • Featured image by geralt, under a Pixabay licence
  • When you leave a comment, you’re agreeing to our Comment Policy

Michael Schrage is a research fellow with the MIT Sloan School Initiative on the Digital Economy and the author of ‘Recommendation Engines,’ MIT Press ‘Essential Knowledge’ series.

 

 

 

 

About the author

Michael Schrage

Michael Schrage is a research fellow with the MIT Sloan School Initiative on the Digital Economy and the author of ‘Recommendation Engines,’ MIT Press ‘Essential Knowledge’ series.

Posted In: Technology

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.