Homepages are the digital town square for news Web sites. It’s where readers find the latest or most important news of the day, and for most readers, it’s their first stop.

Keeping this page fresh and inviting for readers requires almost continuous optimisation. Historically, this critical function has involved subjective editorial decisions and manual processes. However, emerging data science solutions can provide data-informed ways to enhance content curation and predict article performance.

Dr. Sarah Van Cor-Hosmer, lead data scientist at MediaNews Group, developed two independent models to help digital journalists: ROBIN (RandomForest Optimization of BlueStar Index Node) and CHIRP (Content Hierarchy for Intelligent Repopulation and Personalization). Both are AI-powered tools designed to refine homepage content management.

While ROBIN places the best articles on homepages, CHIRP removes the worst, with a goal of using data to help curate the page ethically and responsibly, where editors always have the final say. 

How it works

ROBIN operates as a performance alert system based on historical click data from the homepage. Every 10 minutes, ROBIN scans our CMS for newly published stories and processes them through its specialised ensemble learning algorithm to predict whether they have a high probability of exceptional homepage performance.

Stories are classified into two primary categories: “white,” indicating an expected performance in the bottom 75%, or “blue,” signifying a prediction of performance within the top 25%.

For news organisations with sufficient data, ROBIN can also identify “gold stars” — exceptionally high-performing outlier stories. Only “blue” and “gold star” predictions are disseminated as alerts to newsrooms through widely used platforms such as Slack.

The data indicate articles designated as “blue” by ROBIN achieve 2.5 times higher view counts compared to unlabeled stories, a statistically significant difference.

Newsrooms such as Southern California News Group — which includes the Orange County Register — have adopted ROBIN, with digital journalists utilising its “blue star” recommendations for homepage curation.

ROBIN’s value is its capacity to predict the stories that yield the most impact, rather than just reflecting current popularity.

CHIRP complements ROBIN’s predictive capabilities by acting as a refresh alert tool, notifying digital journalists when a homepage article’s performance has reached diminishing returns by analysing homepage click data from the preceding three months. A CHIRP alert is triggered when an article demonstrates a 90% probability of sustained below-average performance. 

CHIRP’s dashboard provides actionable metrics for busy digital journalists, including first-hour views, story position (e.g., hero position, bottom four, latest headlines), performance recommendations (“good to go” or “remove”), and the percentage a story has fallen below average. It also suggests a top-ranking story for the hero position, which typically generates the highest volume of clicks. 

Compelling results

Van Cor-Hosmer’s implementation of ROBIN and CHIRP has shown considerable effectiveness. Machine learning has consistently demonstrated its ability to surpass human curation in terms of accuracy and effectiveness in comparative analyses.

Historical tests revealed AI models attaining 80%-90% accuracy, whereas human producers achieved approximately 65%. This established performance record contributes to trust in the tools, although human oversight remains essential for specific scenarios. 

However, we are also aware of the biases inherent in homepage recommenders and automation, including the potential for ethical concerns and “news bubbles” that may arise.

MediaNews Group Senior Editor and AI Committee Chair Kristyn Wellesley points out the importance of a manual override option for newsrooms for multiple reasons, including managing breaking news situations and surfacing essential journalism important to all readers, including those from historically marginalised communities.

While stories about the latest city council meeting might not historically generate a large number of pageviews, it’s important for readers to have access to that story from the homepage so they can be informed voters and residents. Editors should be able to handpick articles for the homepage that may not gain the highest engagement metrics but are still important to their readers and community. 

The future: automation and personalisation

Currently, ROBIN and CHIRP are recommenders that generate alerts for human intervention. Automating this process could yield substantial efficiencies, particularly for homepages or section fronts that are not frequently updated.

Automation is also essential for effective personalisation. Without it, it would be virtually impossible to create and manage multiple versions of a homepage for diverse user groups without significant resources. The future vision for personalisation encompasses training individual AI models for each user group to ensure highly tailored content recommendations.

By leveraging user engagement data, news publishers can craft highly relevant user experiences. Imagine personalising the Denver Post homepage for a dedicated Broncos fan. News organisations that integrate these intelligent tools could lay the foundation for a future where news consumption is more intelligent and deeply personalised, fostering stronger connections with their audiences.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .