Testing different headlines for an article is one of the most effective actions a newsroom can take to ensure articles are read more widely.

The (loyalty) click-through rate (CTR) not only highlights the appeal of one particular story over others, but a clear title also sets the right expectations for the target audience.

And, once you’ve discovered the best headline option, it’s not just your Web site that benefits: It will also be displayed across other channels, such as social media and search engines, giving you a much-needed edge over the competition.

A previous data study by smartocto revealed winning headlines receive, on average, nearly 20% more clicks than the original, even when including articles where the original headline performed better. These results show an analysis of 9,000 headline tests across 57 media titles; these always take loyalty into account.

A collaborative study

The new study was presented at the recent INMA Media Subscriptions Summit in Amsterdam. The participants were:

• Vrije Universiteit Amsterdam, which led the research.
• Publisher DPG Media, which provided access to data from four newspaper titles.
• Smartocto, which provided the headline testing system called Tentacles.

Associate Professor Luuk Lagerwerf of Vrije Universiteit categorised the headlines by assigning values to words used in them and comparing their performance against headlines without those words.

“Editors typically rely on their experience to craft headlines with specific effects in mind. I wanted to determine the precise impact and see if we could define it in an academic manner,” Lagerwerf said.

Key findings

One research question was whether quotes in a headline lead to a higher click-through rate. The answer, according to this study, is no. Similarly, posing a question in a headline does not necessarily yield better results when looking purely at clicks.

Another notable finding is that there were differences between the media brands. For instance, focusing on personalisation worked well for only two of the three regional titles included in the study.

There are also other clearly measurable positive effects, which news publishers can use to their advantage by following a few key recommendations:

• Try to include as much detail as possible in a headline.
• Mention the main characters from the article.
• Use a narrative headline — one that tells a story.

Take, for example, the winning headline in the headline test below (indicated by the red circle).

Aggregating the results from all four titles, the most significant factor influencing click rates was the presence of negative words in a headline.

The role of negative headlines

“No news is good news” is a well-known saying in journalism.

However, data analyst Roy Wassink from DPG Media emphasises this does not mean journalists should seek out negativity just to make a story perform better.

“We also look at other metrics beyond CTR. For example, because we focus on loyalty, we know too much negative news can result in fewer returning visitors over time,” Wassink said. “It is also crucial that the headline aligns with the article’s content to achieve better completion rates.”

During his presentation at the Subscriptions Summit, Wassink showed an example where a negative headline did not match the positive lead. The result? Many readers abandoned the article early. A headline, after all, is about expectation management — the expectation set by the headline must be fulfilled in the article itself.

Next steps in research

DPG Media plans to delve deeper into the data to uncover new insights.

In the future, smartocto will introduce a feature on the front end of news sites that visualises the balance of different user needs within headlines. Wassink also advocates for a greater focus on loyalty in headline testing. He believes smartocto’s existing loyalty click-rate metric can be further refined and expanded, to make sure the conclusions can go beyond clicks.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .