Telegraph creates new scoring metric for audience engagement and retention

By Dan Silver

The Telegraph

London, United Kingdom


When used properly, metrics are powerful tools for helping editorial staff make the decisions that will ultimately achieve critical business objectives. At The Telegraph, that means hitting CEO Nick Hugh’s publicly stated target of 1 million paying subscribers from a pool of 10 million registered users by the end of 2023.

When not used properly, however, bad metrics lead to bad decisions.

When I first arrived at The Telegraph in 2017, the KPI used in the newsroom was pageviews. As anyone who fought the clickbait wars of the early 21st Century can attest, that led to some bad decisions indeed.

Thankfully, The Telegraph’s pivot first to registration, and then to digital subscription, also heralded a change in the way the newsroom thought about audience and user behaviour. For the first year or so, the metric used most in the newsroom was subscriber conversion. While acquiring new paying customers was obviously important to the business, these efforts that focused on pushing people through the door — without checking to see if there was a hole on the other side — was inherently inefficient. Every editor in the newsroom could rattle off their top 10 converting articles from the past 24 hours, but not many could speak knowledgeably to their desk’s subscriber retention levels or churn rate. 

The Telegraph’s freemium content access model added further complexity.

The bulk of our day-to-day news output, for example, was open and free for all users to access. Using a traditional “last touch” attribution model meant it was impossible for that content to drive direct acquisitions, and thus there was a danger of it being undervalued or, worse, ignored (by non-paying readers, not our actual subscribers).

Also, only senior editors had the power to decide whether or not a paywall appeared on a piece of content, which meant the majority of staff in the newsroom couldn’t influence their KPI in any meaningful way. 

These issues were very much on my mind when I arrived at INMA’s 2018 Consumer Engagement Summit in Miami. I attended presentations by Gadi Lahav of Financial Times and Karl Wells and Carla Zanoni from The Wall Street Journal on, respectively, how a North Star Goal can help develop better products, and how subscriber-first engagement metrics can increase retention and reduce churn.

Inspired, I returned to London determined to change the way The Telegraph’s newsroom used data to make decisions and inform strategy. 

From that, Project Habit was born.

Building a better metric

For much of last year, I felt like Billy Moneyball Beane railing against Major League Baseball’s institutionalised belief in outdated and misleading analytical structures and data sets. Editorial colleagues were universally open to theory surrounding engagement and retention but understandably reluctant to implement practical change, while acquisitions dominated newsroom dashboards, reports, and analytics conversations.

As neatly encapsulated in an excellent INMA session by Der Spiegel’s Head of Subscription Growth Wiebke Meeder, the key to winning the hearts and minds of the newsroom is fundamentally changing that narrative to better reflect how subscription models rely as much on loyalty as they do conversion. The challenge is doing so in a way that is simple and intuitive enough for everyone in the newsroom not just to grasp, but also to influence.

My brilliant colleague in Insight & Analytics, Emma Wicks, and I imposed other restrictions as well. We wanted to develop a live metric, which meant using data we could measure in real-time. And it needed to pass most, if not all, of the five characteristics of a good metric identified by The Guardian’s Chris Moran: be measurable, actionable, reliable, readable, and relevant. It also had to conform to that unwritten rule regarding a single KPI.

STARS has been integrated with The Telegraph’s proprietary live analytics platform, Pulse, to provide a daily snapshot of how the site is performing in terms of acquisition, engagement, and retention.
STARS has been integrated with The Telegraph’s proprietary live analytics platform, Pulse, to provide a daily snapshot of how the site is performing in terms of acquisition, engagement, and retention.

After a couple of months of toiling at the analytical coalface, we eventually unearthed STARS — the Single Telegraph Acquisition and Retention Score. Bending the rules ever so slightly, STARS is actually a super-metric derived from three feeder metrics that identify and reward content that:

  • Converts new subscribers (Acquisition).

  • Helps keep them subscribed (Retention).

  • Resonates with registrants and new users — the potential subscribers of tomorrow (Engagement). 

The special sauce of STARS is a formula that weighs the contribution of those feeder metrics based on the relative ARPU of new and loyal subscribers, while also factoring in display advertising revenue, and ensures that no one component unfairly dominates the others. 

Unveiling STARS power

STARS has quickly been embraced as the default metric in the Telegraph newsroom. It powers the daily reports and self-service data hubs used by all editorial staff, informs the daily, weekly, and monthly targets allocated to all desk editors, and is even measured in real-time by The Telegraph’s proprietary analytics platform, Pulse. 

Pulse also allows Telegraph editors to drill down into the performance of individual articles to monitor how their contribution to the business changes over time.
Pulse also allows Telegraph editors to drill down into the performance of individual articles to monitor how their contribution to the business changes over time.

To drive that level of adoption, Emma rebuilt our already versatile self-service Google Data Studio reports with simple but eye-catching data visualisations that emphasise how the three strategy pillars correlate to the overall business strategy. Those with a keener interest could then cut the data however they saw fit for deeper dives. 

We also analysed 18 months' worth of historical data to devise realistic scales of what “good” looks like for different content verticals. For example, our paywalled Politics section operates on a different scale than smaller lifestyle sections such as Gardening.

STARS is very much in its infancy and, as such, we’re still wading through the wealth of insight it has to offer. However, it has already been shining a light on the way that different content types serve different purposes. 

Pre-STARS, articles with high conversion rates dominated discussion and content strategies. Now there is a new focus on content that encourages existing subscribers to return to our platforms or that drives high engagement with registrants and anonymous users.

Even in these early days, there are already signs that subscriber survival rates are moving in the right direction. Arguably its biggest success, though, is getting the newsroom thinking about the importance of engagement and retention and how those aspects relate to the content they produce — for many of them, the first time they’ve approached content in that way. 

The Telegraph’s subscription success is now written in the STARS.

About Dan Silver

By continuing to browse or by clicking “ACCEPT,” you agree to the storing of cookies on your device to enhance your site experience. To learn more about how we use cookies, please see our privacy policy.