The JournalismAI Fellowship began in June 2022 with 46 journalists and technologists from news organisations globally collaborating on using artificial intelligence to enhance their journalism. At the halfway mark of the 6-month long programme, our Fellows describe their journey so far, the progress they’ve made, and what they learned along the way. In this blog post, you’ll hear from team What’s There, What’s Missing.

The use of algorithms and automation in newsrooms can serve various purposes: the analysis of large amounts of data, for example, or making work easier for repetitive tasks. This is where a joint project of Bayerischer Rundfunk, Mitteldeutscher Rundfunk and the digital agency ida, a subsidiary of MDR and ZDF Digital, comes in.

As part of the JournalismAI Fellowship of the London School of Economics and Political Science, journalists, developers and machine learning experts from the two public broadcasters joined forces in June 2022 to work on an ambitious project.

Both media houses have already gained experience in the field of automation in journalism, BR with its own AI + Automation Lab and MDR – together with ida, among others – in the implementation and development of a model for automated election reporting.

The idea

BR and MDR are public broadcasters with a mandate to enable users to participate in democratic discourse – by covering a wide range of topics, perspectives, and opinions and engaging in conversation with our readers. Thousands of comments are published daily under the reports on the websites of BR and MDR. They are read by our editors, published, and spark conversations on social media, radio, and TV. They are also an important way for our readers to react to journalistic content.

For the editorial teams, however, it is challenging to sift through this feedback and use it for further work and discussions due to the large number of comments and contributions. At the beginning of the fellowship, this led us to wonder whether it may be possible, with the help of algorithms and machine learning models, to automatically identify topics in the articles and in the comments, and to find out where editors and individual authors are directly addressed by our readers, to help them follow up on the relevant comments.

The advantage of technical support in these tasks is that sorting comments and user contributions helps editors to react faster and understand their audience better. With thousands of comments daily, it can help editorial teams respond to questions and reactions in a timely manner and keep track of the different topics.

Ultimately, such an algorithm can also help identify issues and aspects of a topic that have not yet been covered in the journalistic articles. Editorial offices can thus not only communicate better with their users, but also expand their content offerings in order to better fulfil their information mandate.

With this idea, BR, MDR and ida joined the JournalismAI Fellowship to develop a solution together until the end of the year. In doing so, we benefit from the experience of the other participating teams from international media houses, who are also working on solutions for how algorithms and automation can help them in their work and with journalistic issues.

Fail early: Developing a useful feature is difficult

Based on the previous question, the title of the joint project was created at the beginning of the fellowship: What’s there, what’s missing.

At the beginning of the project phase, we discussed various application scenarios that could be realised with the help of algorithms, or more precisely, so-called topic modelling. In the field of machine learning, topic modelling is a procedure that enables automated content analysis and pattern recognition in texts.

A useful principle in the conception of projects, especially in the case of elaborate and complex developments, is the method of rapid failure. Errors are bound to happen, but in order to catch them as early as possible, one tries to find the causes for them at the very beginning of the development.

For this reason, extensive user interviews were conducted with social media editors right at the beginning of the project to find out what the editorial teams at BR and MDR really need in terms of support. In addition, regular feedback discussions with colleagues were integrated into the development from the beginning.

It turned out that most of them would like to have tools to process the thousands of comments daily more effectively and thus save time. For example, it is extremely important for the editorial offices to be informed promptly when authors are addressed directly in the comments, when errors are pointed out, or when suggestions for further research are made by the readers.

On the one hand, because these comments are needed for our own internal editorial development of topics, on the other hand, because it is important to be able to answer users quickly and concretely and to enter into an exchange.

In order to achieve this goal, a special NLP model is trained to recognise the most diverse forms of addressing the editorial team in thousands of comments. The prerequisite for this is to read through more than 7,000 comments from BR and MDR so far, and to mark in them the exact places where the editorial offices are mentioned.

The status after four months: The path is the goal

Where does the joint team of BR, MDR and ida stand now after four months in development? It has become clear that it is very difficult to find really useful applications. But it is definitely worthwhile to set out together, because the cooperation and learning from each other has been greatly facilitated by the JournalismAI team.

The technical solutions found and models trained are to be documented and also shared and discussed with other media houses and broadcasters as a possible standard – for example, with regard to the technical definition of which model was best able to recognise mentions of the editorial team in user comments.

In addition, inspired by the monitoring of the other Fellowship projects of the international teams, teams with similar interests have quickly come together.

In this way, the project is also intended to fulfil the mission of public service broadcasting with regard to the provision of information. The findings from the fellowship are to be used to communicate better with the users and to gear the content better to the needs of the audience.

Team What’s There, What’s Missing is made up of:

• Jörg Pfeiffer, Product Manager, Bayerischer Rundfunk (BR)
• Philipp Gawlik, Language Engineer and Computational Linguist, BR
• Martin Paul, Chief of Service, Journalist, MDR
• Jaime Avalos Mongil, Data Scientist, ida

Do you have skills and expertise that could help the team? Get in touch by sending an email to Fellowship Manager Lakshmi Sivadas at lakshmi@journalismai.info.

This article was first published (in German) on the MDR website.

JournalismAI is a global initiative of Polis and it’s supported by the Google News Initiative. Our mission is to empower news organisations to use artificial intelligence responsibly.