“Are we doing the right stuff?” That’s the core question for every newsroom with the ambition to satisfy audiences today.

In the beginning there were pageviews. And unique visitors. This was perhaps perfect for measuring mass-produced journalism for maximum traffic volume and a business model based on eyeballs and advertising.

But what kind of story do those numbers tell us in the reader revenue-driven world? This isn’t really the sharpest way of getting a grip on what actually creates value for our paying subscribers.

So, what can we do?

Some years ago, editorial analysts and the digital subscription knights of the newsrooms began to dig deeper into the engagement and loyalty metrics.

If we want to find out what kind of journalism is really triggering the optimal subscriber behaviour for a reader-revenue business, what should we look at? And, how should we make that knowledge easily accessible for reporters and editors in the newsrooms? If we wish to encourage readers to read and watch a lot of our journalism, and also return often, what metrics should we use to create that habit?

These questions are vivid in my daily work at the conglomerate NTM in Sweden. 2020 is the year of redefining our newspaper’s strategy on reader revenue and heavily level up. Under our project “Let’s get ready to double,” we are revising everything.

In my mindset, listening and “copying with pride” are excellent ways to accelerate if you begin in a position a bit behind the rest of the business. The north star of the Financial Times, the RFV score, is well known to most readers here, but I’m curious to see what more is out there today.

So, let’s take a look through the window and reach out to our Nordic INMA diaspora to see if we can find some other “holy engagement grail” out there.

Amedia

Norwegian Amedia is focusing heavily on leveraging reading of its premium content, “plusartiklar.” The primary KPI is measuring the number of subscribers who have read the piece. Eivor Jerpåsen, head of the editorial development, explains they count a “read” not a click. The user must have read the article for at least 10 seconds to be counted. The company also measures how well every article engages different age segments.

Amedia also uses an “engagement index.” The index combines different parameters, such as: how often readers visit, how much they read, how long they read, and how much time they have subscribed. The index goes from 0 to 1. If you have an engagement over 0.6, the chances for remaining a subscriber are high.

Amedia then uses this index to see if changes in how staff members write articles, develop products, send push notifications, compose newsletters, and so on contribute to an increase in subscriber engagement. The index can also be used on articles, but that has not been implemented so far in the newsrooms.

Bonnier News Local

Robin Govik, head of development at Bonnier News Local, said the company currently is using a “reader value.” This KPI used in the daily work in the newsrooms is a mix of subscriber pageviews and spent time. 

Svenska Dagbladet 

The Swedish Schibsted-owned daily newspaper Svenska Dagbladet also uses an aggregated value. Editor-in-chief and CEO Anna Careborg said: “The ‘reader points’ are a self-developed combination of conversions, pageviews, interest (CTR) reads, and quick exits. We use it to identify the best articles for our reader revenue business.

“We also keep our eyes on net gain in subscriptions, unique visitors from subscribers, and spent time when we evaluate. To track the engagement of our digital subscribers, we are also using our ‘compass,’ measuring the recurrent engagement per week.”

Ekstra Bladet

Ekstra Bladet, a Danish tabloid with a freemium product, focuses both on reach and reader revenue. And the company has a different approach. It is not trying to invent some aggregated score.

For the free material, the company is focusing on pageviews. For the paid content, is focuses on conversions. Kasper Worm Petersen, head of strategy, explains why his team is using these kinds of basic metrics: “We spend a lot of time doing correlation analyses to see which simple KPIs best reflect the complex strategic goals. When it comes to the free articles, we can see that pageviews correlate positively with both visits, time spent, and users, so therefore we continue to look at pageviews.

”On paid artices, we can see that sales correlate positively with the share of the customer base that reads the articles, pageviews, and time spent, so therefore we continue to look at sales,” he said.

“These are simple KPIs that make it easy to understand the feedback and assess whether the article was a success or not. My experience is that journalists and editors build up a good understanding of what works and does not work by constantly orienting themselves in relation to the simple KPIs. They themselves know which variables they have changed from article to article and thus they build up a strong experience in relation to what constitutes a well-performing article. When requested, our data team helps with additional in-depth analyses to find the larger patterns.”

However, Petersen noted KPIs can only ever inform evaluations on article performance. The final conclusion has to be made by editors and journalists taking the editorial values, context, and strategy into account. “Basically, I think KPIs need to be ‘actionable.’ In my opinion, a well-functioning KPI is one that provides direct, transparent, and easy-to-understand feedback that can inform your practice.”

Dagens Nyheter (DN)

Dagens Nyheter, a Swedish daily and part of Bonnier News Local, is often highlighted as a successful media company in the “economics of journalism.” The company has done comprehensive work on KPIs.

Martin Jönsson, head of editorial development said: “We started four-and-a-half years ago at DN and have changed basically everything in our working methods: our business model, how we are organized, and what tools we use. Analysis and KPIs have been incredibly important in this.”

The company builds tools available for both news desks and reporters, and it makes sure staff understand why articles do or do not perform well. Additionally, DN ensures everyone has the same understanding on what we describe as a good article.

“We want to get away from vanity metrics — the one-dimensional KPIs such as pageviews, number of unique, or shares that previously defined success,” Jönsson said. “We are now focusing on the bigger picture and steer more toward qualitative KPIs. Those that, based on the subscription business model, most clearly indicate success.”

He has been working on a loyalty-focused project (“strengthen the DN habit”) for the last two years, getting readers to visit more often and use more of DN’s content for a longer period of time. Through this, the team developed new loyalty measures, which are the most important measurement.

“If you look at the individual article, the most important thing is total reading time. It indicates that we have reached out widely and that the readers have actually read. That’s the finest KPI list you can be on top of in our newsroom,” he said.

DN has also developed something similar to what Financial Times calls “quality reads,” introduced by Renée Kaplan a year ago. This measures the proportion of people who have read at least half of the article.

“If that number is too low, you have problems,” Jönsson said. “Maybe the article is too long, has too few visual elements, or is just too boring.”

Additionally, DN is using a total engagement score. This is an index, reminiscent of Die Welt’s article score, where the company sets percentages based on various factors. In DN’s case, reading time and purchases are ranked highest followed by social media shares while pageviews over the last hour is the lowest indexed.

The team is using this to see if articles are creating high engagement. This is a qualitative index on a scale of one (lowest) to five (highest) stars. Two is a normal ranking and everything over two stars is performing better than normal.

“Of course, like all other scores, it’s a bit of a blunt tool, but it is great for our news desk,” Jönsson said. “They follow pageviews per minute, but this gives them a possibility to also see the qualitative measures.” It also helps editors decide whether to boost certain articles.

“Overall, we have moved toward what I think gives the most effect: When you can work with individual reporters, news desks, and articles. Where you can see how this or that article worked and understand what you can do to make it work better next time. The reporters must understand the driving forces behind the performance — the right publication time, how the whole package meets the reader. Is the headline good enough? Has the article been shared in social and distributed in newsletters in the right way?”

Jönsson notes the basic challenges is how to actually measure quality: “My definition of quality is if a reader reads for a long time (preferably to the end), shares the article with friends, returns as a reader, and reads even more. Then you can on fairly good grounds assume that it is a satisfied reader.

“If it is a reader who gives up early and does not read several articles and does not share anything, then one can assume that it is a dissatisfied reader. Then one must try to define the KPIs as thoroughly as possible based on that view. When the behaviour we want occurs, then we know that it is quality. And hopefully that means they do not cancel the subscription.”

NTM

In our transformation at NTM, we are currently experimenting with an “article value” based upon pageviews from subscribers, spent time, completion rate to 80%, and conversions. Listening to our colleagues makes me even more curious to explore it further.

However, I am also a bit pensive on how to ensure reporters will know how to act on this information. Regardless, the most important part of working toward a goal is to walk the talk, and get editors and reporters into structured conversations about the results to take action on it. I’m also quite convinced they need to get the numbers close to their daily work, so integrating it into our reporter tool will be a high priority for us.

I don’t know if you actually could find a holy grail here, but hopefully this provides good thoughts on KPIs and strategy.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .