It’s estimated that by age 13, online ad companies have collected over 72 million pieces of information about each child. If that fact doesn’t demonstrate that it’s time for a change in approach, it’s unlikely that anything will. For www.parenting.digital, Valerie Steeves discusses her hesitation about the global push to adopt the UK’s Age Appropriate Design Code and the dangers of giving children and parents limited data protection rights to manage the impact of the surveillance economy.
A number of years ago, just as the eQuality project team was looking for a way to reverse engineer the algorithms that are increasingly shaping childhood, Mattel released Hello Barbie. The doll had a two-way microphone that captured everything a child said as he or she played with the doll and then sent that data over Wi-Fi to a third-party company called ToyTalk. ToyTalk’s server then ran the child’s speech through an algorithm that would select a phrase that the doll would then “say back” to the child.
Hello Barbie was a gold mine for us because Mattel published Hello Barbie’s lexicon, a list of over 8,000 phrases that could be selected by the algorithm and “spoken” by the doll. Although the child’s input could vary, the doll’s lexicon gave us a detailed picture of just what Mattel was trying to accomplish in its dialogue with children. Although the doll was marketed as an ideal “best friend” who could listen and respond to children, the lexicon was designed to constantly shift the dialogue to either collect data from the child or to encourage the child to “feel” receptive to the Mattel brand. In many ways, the doll was a trojan horse that constantly (re)inserted a customer relations management program into child’s play.[1]
So, I have mixed feelings about the global push to adopt the UK’s Age Appropriate Design Code. The Code uses standard data protection provisions to try to give children and their parents more control over their data. But – and this is the most important part – it also adopts the language of child rights, attempting to encourage designers to create products, like networked toys and personalized education apps, that are “in the best interests of the child”. So far, so good.
Actually, so far, so great. I remain convinced that the most important policy document to be created to advance children’s online privacy since 1970 is the UN Convention of the Rights of the Child’s General Comment no. 25 (2021) on children’s rights in relation to the digital environment. The comment is a nuanced and important statement of the need to ensure that tech companies create apps that not only protect children or provide them with services but also enable them to participate in the networked society in ways that deepen their understanding of themselves and their culture.
The UK Children’s Code is a hopeful step in this direction precisely because it inserts some of the language of the Comment into legislation, but I worry that that language will be interpreted in ways that weaken its impact. I think back to the push for legislation in the US in the late 1990s. Media scholars and child rights advocates argued that inserting children into the surveillance economy by collecting their data and inserting commercial content into their play spaces so children would “feel” good about brands – exactly what Mattel did with Hello Barbie – violated children’s dignity and manipulated them for profit. Because of this, they called for these practices to be banned. Instead, the US Congress passed the Children’s Online Privacy Protection Act (COPPA), in essence giving children and their parents limited data protection rights to manage the impact of the surveillance economy on the young.
The logic adopted in COPPA was easy to pass precisely because it worked for the corporations that were designing apps for children. Data protection assumes that the online environment brings great benefits to young people with some (manageable) risks, so a tweak here and there is all that’s needed to fix any problems. But our research has consistently shown that it’s the commercial agenda behind these apps that creates problems for young people. In other words, it’s the profit-driven imperatives and the need to create “sticky” apps that can collect huge amounts of data to feed into the surveillance economy that set young people up for online conflict and shut down opportunities for creative and democratic interactions.
Let’s look at Hello Barbie. The lexicon is designed to direct children to games that enable Mattel to collect detailed information about the child – for example, the doll encourages kids to keep a dream diary and to share their nightly dreams with Mattel. But it also collects information about the child’s family. For example, the Family Store game asks the child to identify what movies family members like best. The terms of use also require that the child not change the doll’s clothing or move around too much during play because that will make it hard to Mattel to collect the data. The whole idea of the doll is to generate data for customer relationship management purposes, so it’s hard to see how the doll can be privacy-respecting even if parents can delete snippets of recorded conversations. The problem isn’t that there aren’t enough tools to put parents and children in the driver seat. The problem is that corporations are using toys to collect detailed personal information from children so they can commodify that information for profit.
But Barbie’s lexicon is also an excellent example of how quickly these algorithmically-driven apps can embed discrimination into a child’s world. For example, Barbie likes to chat about holidays but its dialogue around Diwali and Hannukah is pedantic at best, “teaching” the child about the basic spiritual meaning of the holidays and providing lots of exit strategies if the dialogue falters. Conversations about Christmas and Thanksgiving don’t mention spirituality at all, instead focusing on the “joys” of shopping. And in all 8,000 lines of dialogue, there is no mention of Eid, virtually erasing Muslim kids from Barbie’s play. All of this works to try to shape the kind of “person” the child can be, making some identities easier to inhabit and erasing others.
Designers may try hard to comply with the Code and develop apps in the child’s best interests, but my guess is that they’ll continue to do this without interrogating the real problem: using young people as fodder for a surveillance economy. If you’d like to see what this is going to look like, take a peek at Meta’s latest design initiative designed to apply the UN Convention on the Rights of the Child. Once again, any real change is stymied and the focus is put on developing apps for parents to better supervise their kids instead of shutting down the collection of children’s information.
The inclusion of child rights language in the Code is a promising start. In the Canadian case, the federal Privacy Commissioner has used a rights-based clause in Canada’s private sector privacy law limiting surveillance to purposes that a reasonable person would consider appropriate to push back against micro-targeting of children. The language in the Code can only strengthen the Commissioner’s approach. But to fully ensure that the Code is implemented in ways that meet children’s needs, legislation must give oversight bodies strong powers to ensure that apps are limited to purposes that truly serve children’s best interests. And sometimes, that’s going to mean just saying “no” to the collection of children’s information. In the long run, empowering a rights-oriented body to stop certain kinds of collection may be the only way to make sure we live up to our commitment to create an online world where children can thrive.
First published at www.parenting.digital, this post represents the views of the authors and not the position of the Parenting for a Digital Future blog, nor of the London School of Economics and Political Science.
You are free to republish the text of this article under Creative Commons licence crediting www.parenting.digital and the author of the piece. Please note that images are not included in this blanket licence.
Image credits: photos by Ksenia Chernaya and Pavel Danilyuk on Pexels
[1] For the full analysis, see Valerie Steeves, (2020), A Dialogic Analysis of Hello Barbie’s Conversations with Children, Big Data & Society (7). 1, available at: https://journals.sagepub.com/doi/full/10.1177/2053951720919151
