Until 2019, staff at The Sydney Morning Herald and The Age newsrooms had to access multiple data tools in their search to answer what should have been very simple questions. Instead of gaining useful insights quickly, editors found themselves wasting valuable time asking where they needed to look for information or doubting inconsistent numbers.

We’ve spent the past year consolidating the various data sources available to our editorial team. We listened carefully to the questions editors were asking, considered which metrics would have the biggest impact on our content output, and examined how we could create more of what our audience wanted.

We pride ourselves on being newsrooms dedicated to attracting and retaining subscribers. So, learning that just 40% of our content made up 87% of what our subscribers were reading online was a confronting but powerful lesson.

No one wants to focus too hard on their weak points, but we discovered the newsroom could learn more from the 60% of content failing to find a meaningful audience than the content already working hard for us. The problem was editors had no easy way of identifying underperforming content.

In the quest for answers, our topic editor dashboard (TED) was born and with it a new way of communicating our editorial strategy to the newsroom.

Custom measure of success

Our data team created custom benchmarks for teams, journalists, article formats, and content themes (tags). They are defined by four key metrics: subscriber pageviews, best prospects (the group of people with behaviours we’ve identified make them most likely to subscribe) pageviews, average engaged time, and completion rate. TED is the visual representation of these benchmarks.

Teams are defined by reporting lines so editors have a view of content they commission directly.

Tag benchmarks help editors understand the context and performance of content when different teams are producing articles about the same subject. For example, multiple teams create journalism that is tagged “coronavirus pandemic.”

Word count benchmarks help editors understand the performance of articles of different lengths and in different templates, such as short articles, features, explainers, and investigations.

Benchmarks represent a quartile-split of a historical 12-month reporting period. The four quartiles are split into coloured zones, where green represents the top 25% benchmark and red represents the bottom 25% benchmark.

Team, journalist, and word count benchmarks are static. Keeping benchmarks static allows us to measure improvement over time. Tag benchmarks are rolling in order to capture newly created tags and understand the content in the context of current subscriber engagement.

Establishing benchmarks was never about ranking journalists or teams; it’s about being able to identify what content didn’t engage subscribers as we expected and why.

Learning from the red zone

Before benchmarks, editors had no clear definition of what success for their teams looked like in a subscription-driven newsroom. We had to connect day-to-day content commissioning and distribution to the bigger picture of subscriber acquisition and retention.

Historically, when the business goal was reach, editors knew what to do. For them, it was simple to connect the concept of “lots of pageviews” to what stories they should write and how to distribute them. But it also led to clickbait content that wasn’t particularly on brand or worth paying for.

Today, our editors focus on what subscribers and best prospects read and how they interact with our content.

As an example, we know our subscribers and best prospects navigate to our content from the homepage above any other referral method. We also know in-article referrals for these user segments are higher than Google, Facebook, and Twitter.

This is an empowering insight for editors because it means, for the most part, they have control over what our most valuable audience sees and when they see it. Home page promotion, related article widgets and placement, headlines, and image choice are all things we can control and track.

It’s not as simple as recognising an article flopped and making an immediate decision to stop writing about that subject. For every coronavirus story at the top, there are at least 20 at the bottom. But if a story is important, it’s our job to make it interesting.

Sometimes editors recognise it’s as simple as fixing the headline, pitch, publish time, or distribution. Other times we work together to identify patterns in content that are at odds with our audience’s values or interests.

With that lens, incremental updates from multiple teams become in-depth explainers with context. Unnecessary embargoes are lifted. Headlines are workshopped for the homepage in the same way they are for page one in print. And, ultimately, we create more content worth paying for.

Do less, make it better

We don’t have time or financial resources to waste on content nobody reads. Our goal is to cultivate a culture of learning from our mistakes and telling important stories in the most engaging way possible.

To achieve this required a reset. We’d tried (and failed) to launch new data products with the newsroom in the past. Without accountability, initial enthusiasm was overtaken by the daily grind of the news cycle. On top of that, editors had never been asked to articulate their content strategy to anyone outside their team.

With that knowledge, we made red zone analysis compulsory and gave editors six weeks to present their findings back to their managers and peers. At first, they were sceptical. But as the weeks went on, they became more excited by what they were discovering.

Every editor found the content in the green zone backed up decades of gut feelings, while the red zone was full of opportunities. Together they workshopped solutions and set goals to produce less, but make what they produced better.

Editors presented their findings throughout the second half of 2020, and they’re already seeing results. Our lifestyle team produced 50% fewer articles last year when compared to 2019. As of September, the team had no content appearing in its red zone. The number of subscriber pageviews on its worst-performing article is now 40 times higher than it was before TED was introduced.

Bringing editors on the journey has ensured their expertise is reflected in our content strategy. We don’t just celebrate success, but rigorously interrogate how we tell a story. After all, every minute wasted creating content nobody reads is a minute that could be spent creating content worth subscribing to.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .