In a previous INMA article, I wrote about how we are experimenting with personalisation in the news experience at Mediahuis. Another text I wrote described how we streamlined our newsletter creation process. I received a lot of positive feedback on both, so I thought it would be interesting to combine both topics in this piece.

Why create a personalised newsletter?

Mediahuis implemented successful paid content strategies a few years ago. One of the challenges we faced and still face as a sector is to engage readers with our great content. That was exactly what made us experiment with a fully automatic and personalised newsletter.

Our numbers indicated an unexpected drop in attention time on Sunday, which was, in our opinion, very strange. Didn’t we have a great newspaper on Saturday that consists of a whole lot of stories to publish on the Web site during the weekend? Weren’t we making digital-first and digital-only articles often enough?

What we found out was that we had all of that, but we worked differently during the weekend compared to the week. Fewer editorial staff work on weekends than during the week, which has an impact on some of the tasks performed. For example, during the weekends we did not send as many e-mails compared to what we sent during the week.

This is where we came up with the idea to help the newsroom create an automatic newsletter on Sunday without interference from an editor. The newsletter only consists of premium articles.

Personalisation is better, more relevant than segmentation

Segmentation is often applied in e-mail marketing. However, an e-mail to one segment won’t be relevant for everyone in that segment. That’s because segmented e-mails provide little improvement in relevance despite the hard work they require.

For example, dividing 1 million e-mail addresses into 10 segments of 100,000 people requires the hard work of creating 10 e-mails. There are still 100,000 people receiving exactly the same e-mail, which will only be perceived as fully relevant by a small number of receivers.

Our experiment showed we had higher open rates because the subject line is also personalised, and our click-thru rates (CTR) were higher. Higher diversification of news topics were included in these e-mails.

Personalising Mediahuis’ e-mails

To implement single-person segmentation to Mediahuis’ e-mails, Froomle AI integrated its technology across the brands and platforms. Froomle runs raw data through machine-learning algorithms to recommend relevant articles for readers. The algorithms are fully tunable to a specific use case or a particular business rule.

To optimise the result of the specific use case we add filtering capabilities, diversification, and impression discounting. We also take the user intent into consideration.

Then we A/B test everything.

For this project, we personalised several e-mails for three different brands. We tested hybrid and pure personalisation. Pure personalisation is when the selection of all the recommended topics are fully automated by Froomle technology. Hybrid personalisation is when the content curation comes from both AI automation and journalists.

Lesson #1: Personalised groups outperform popularity groups

To offer more relevant articles, we conduct experiments for each brand. For two of the brands we applied pure personalisation and for one a hybrid model. Each user is allocated to one of the following experiences:

• Experience 1: Showing “most read, not yet read” articles from the given week as a baseline experience.
• Experience 2: The best-performing configuration of our recommender system to date.
• Experience 3: A new and experimental configuration, aiming to become the new “best” algorithm.

After multiple iterations for optimisation, we achieved these results:

• For a “popular” brand, Froomle personalisation consistently outperforms the baseline in CTR with 15% (experience 2 vs. experience 1).

• For a “regional” brand, Froomle personalisation consistently outperforms the baseline in CTR with 19% (experience 3 vs. experience 1).

• Hybrid personalisation at another “regional” brand yields similar results in CTR as a pure personalisation variant (experience 2 and 3 vs. experience 1) while achieving more different clicked articles (diversity).

Lesson #2: Personalisation increases topic diversification

News brands have news for everybody, but they often distribute for the average reader. The gap is in the distribution, when one reader interested in niche topics receives topics selected for the whole segment. These topics could be tagged to specific interests (like opera or cycling), jobs (like Swissport), or region.

Truly personalised e-mails help newsrooms reach the reader with the uniquely relevant content, whether the content is considered popular or niche. They allow news brands to recommend a wider range of articles than a static list can ever realise.

It’s safe to assume that reaching niche readers with relevant niche content allows for a better reader experience, brings readers back to the site more frequently, and creates more loyal subscribers.

Let’s look at our experience personalising e-mails for two brands, one popular and one regional.

The graphs below show the number of different articles recommended to and clicked by all users in a given group. While the baseline only recommends about 50 different articles per week, the personalised (single-person segment) groups recommend more than 800 different articles each week. The second graph shows these users click on twice as many different articles on average (50 articles vs. 25 articles on average).

Lesson #3: First article is critical in e-mail performance

Personalising your subject line is as crucial as personalising the e-mail content. When looking at the total number of clicks-per-position in the mail, it is clear the first article in the e-mail determines whether or not the e-mail will have a good result. The reason for this is also due to the subject line. Therefore, it is crucial to optimise the first article.

Opportunities for improvement

As with every project, we faced some hurdles. I think it’s important to share them with you as well. These three stood out to me:

Outdated articles: As mentioned, we first tested a fully automated and personalised (premium) newsletter without interference of an editor. What we encountered was that — even though we only selected premium content from the past week — some articles were already outdated or at least needed an update on Sunday.

We came up with two solutions to address this: the hybrid concept and negative tagging (noting articles with a short life cycle).

A good editorial mix: Since this is an automated and personalised newsletter, the newsroom was concerned about the editorial mix. Since an important part of the algorithm still relies on “popular items,” this is, of course, something to keep an eye on.

I firmly believe we have this under control for these two reasons: First, we control the “popular” parameter. If it needs to be tuned down, we can do that. Second, we proved the diversification of articles is higher with this personalised.

Good reporting: Because everyone is getting different and personalised e-mail, it’s important to have good reporting to the newsroom. We have some reports in place, but to be honest, I think we still can improve on this. Any ideas are always welcome.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .