Project Overview
Recommender systems are algorithms that suggest items to users based on their engagement and keep users interested in whatever a site continues to recommend. Together with NEMO Kennislink, we researched how to implement a journalistic content recommender system responsibly — that is, without relying on users’ personal information or tracking cookies — in a sandbox setting. The project aims to provide NEMO Kennislink with the tools to build a responsible recommender system that fits them and presents their audience with an enticing reading experience. Additionally, it gives the AI, Media and Democracy Lab an opportunity to work on real-world use cases and conduct applied research to set up a guideline or toolbox on the responsible design of recommender systems.
By applying a sandbox approach, we can safely experiment with new algorithms and evaluate their ethical, legal, and societal implications. This method allows us to mimic the end-user environment and address concerns regarding data privacy and algorithmic bias. Moreover, this collaboration between computer scientists, social scientists and industry partners featured co-design sessions to ensure that the resulting recommender system aligns with both technical feasibility and ethical standards.
Approach
Initially, our researchers Manel Slokom and Sanne Vrijenhoek conducted interviews with key stakeholders at NEMO Kennislink to gather input on the requirements and desired features for a responsible recommender system. The goal here was to ensure the system does not rely on personal information and does not keep behavioral profiles. After an iterative phase of designing the AI-based recommender algorithm based on feedback from editorial stakeholders, the system was implemented into an app designed for studying user interactions with media (Informfully), whose functionality was extended by our researchers to fit this project.
The experiment that followed involved 135 active participants who used the adapted platform on a daily basis to read articles from NEMO Kennislink, and to receive recommendations based on their activity. To evaluate how effective the system was, measures of exposure diversity (in terms of how much the users ended up reading articles that varied across theme, publication date, and author), as well as measures of engagement (measured through user behavior like clicks, scrolls, and reading time, but also through explicit user feedback) were analyzed.
Takeaways
The experiment showed that the recommender system designed for this project effectively increased the diversity of articles users were reading without sacrificing satisfaction and engagement, enabling them to explore relevant pieces across different themes, while also giving some attention to older articles that might have fallen into obscurity after their initial publication. Overall, the researchers suggest that the approach of AI-assisted, content-based recommender systems can act as a complement to existing editorial workflows and decisions, supporting diversified curation of content with high relevance to the reader.
Read the full technical report of this project here.
NEMO Kennislink also published an article series about this project in three parts (in Dutch):
