The idea that e-mail is dead is now well and truly buried. Newsrooms across the world have embraced e-mail newsletters as a major masthead-owned distribution channel. The Sydney Morning Herald and The Age are among the many newsrooms to have doubled down on their newsletter strategy.

Along the way we have asked the following questions: How do we measure their success? And how can we use data to evaluate and improve their performance?

Keep, kill, rebrand

Open rates, list sizes, and click-to-open rates are the staple of newsletter metrics. In early 2020, we used them to audit our existing newsletters to decide which lists to preserve, delete, or rebrand. Some of our newsletters were decades-old and low open rates indicated they were not engaging readers.

At the time, we had 14 newsletters. Of those, we redesigned seven, deleted four, and rebranded three. These decisions were based on insights from audience data highlighting content that engages and converts readers along the subscriber journey.

Newsletter lists are difficult to build. Those with large, active audiences were rebranded while keeping within the theme of the originals. Our Money and Small Business newsletters had previously offered a list of stories from these topic areas. They were replaced by Money with Jess, a narrative style newsletter from our personal finance expert Jessica Irvine. The newsletter’s new focus on practical budgeting tips and Jess’ engaging personality has seen the list increase by 150%, and the average open rate has more than doubled.

What to launch?

With topic editors firmly focused on creating content engaging our most valuable
readers, we wanted to use newsletters to showcase the breadth and depth of our journalism,
serve audience groups with niche interests, and highlight our key talent.

We have launched eight newsletters in the past year: Coronavirus Update, a new Evening Edition, Greater Good (good news stories), The Watchlist (TV and streaming), The Booklist (books), What in the World (which began as a U.S. election newsletter before shifting to broader world coverage), Examine (science and health), and Opinion for the Herald.

Project Engage, an internal research initiative that dove deep into readers who subscribed during the pandemic, found readers wanted newsletters to focus on topics they were passionate about, but that they struggled to easily find on our existing platforms. Books, streaming, and science were all key areas of interest, but given the fast-moving nature of our homepage, content from these areas were either cycled through the homepage or never made it in the first place.

Coronavirus Update began as a daily weekday briefing on the pandemic during the peak of the crisis in Australia and quickly became one of our biggest newsletters. Greater Good was also launched in response to the pandemic when our readers told us they needed stories on hope and community to balance our coverage.

Measuring success

While list sizes and open rates are important, we’re now looking to more detailed metrics to determine the success of our revamped and freshly launched newsletters.

A new dashboard, created by our data and insights team, allows us to easily track metrics that weren’t previously available to editorial.

We can now measure:

• Sign-ups by reader type (such as subscriber vs. visitor and whether the type of subscriber has high, medium, or low engagement).
• New sign-ups and cancellations, per send and over a period of time.
• Overlaps between newsletter audiences.
• Open rates, send counts, and click-to-open rates with heat maps and graphs that visually demonstrate performance and trends, allowing for comparisons between newsletters.
• Click count for each link included in a newsletter.
• How recently readers have opened a newsletter.
• How frequently they open a newsletter.

All of our metrics can be filtered by newsletter and by reader type.

Each newsletter has a different goal that determines which metrics we prioritise.

We pay close attention to the proportion of our lists that are made up of subscribers versus members when determining what content to feature. The goal is not always for newsletters to drive click-through rates, but we know newsletter readers who do visit the site have a much higher on-site journey in that session than those who come from social or search.

Some newsletters aim to retain existing subscribers by surfacing content they are passionate about. Data analysis from earlier this year showed subscribers who open one newsletter per week are 1.64 times less likely to churn, while subscribers who open three newsletters per week are two times less likely to churn.

Others aim to push readers down the funnel to become subscribers. We send millions of newsletters each month, to hundreds of thousands of users. Forty-two percent are subscribers. The remaining 58% are registered users who report high levels of satisfaction with the product. This provides a large pool of prospects for us to tap into.

What we’ve learned

We’ve been able to use our new metrics to make decisions about newsletter content and curation to meet our various newsletter goals.

For example:

• If a newsletter’s purpose is to retain subscribers, we can filter our metrics to see what our low-engaged subscribers who are at risk of churn are opening and most interested in. This informs our decision on what to include and/or prioritise in future sends.
• We can see if particular sends lead to more cancellations than usual and dig into the reasons why.
• We can see if newsletters share a large proportion of readers and therefore avoid repeating content.
• We can check heat maps to see if some days correlate with stronger performance to determine send schedules.

What next?

Our data team has recently provided us with the ability to see what our newsletter readers do after they open a newsletter. Not just how many links they click within the newsletter, but how many articles they view after opening. These new insights will allow us to see if there are content areas our newsletter readers are interested in that could be featured more frequently or prominently within newsletters.

As mentioned, these metrics have shown that subscribers who visit our Web site after opening a newsletter have a higher-than-average on-site journey. For example, subscribers who visit from the Good Weekend newsletter have an average on-site journey of 3.14 articles per session. Readers who visit from the Sport newsletter have an average on-site journey of three articles. This is compared with subscribers who arrive on-site from social, who have an average on-site journey of 1.6 articles.

We continue to refine the use of data and dashboards to help our newsletters meet goals and satisfy our audience needs. Now that we understand this audience more deeply, we’re beginning to explore the commercialisation of newsletters, which will give us another opportunity for value and another goal to measure.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .