It used to be only about the news. Now, you wake up and notice your subscription product is increasingly about different product features. You have a stack with dozens of different ways to offer your content to subscribers: from e-paper to podcasts, from newsletters to videos, from data visualisation to read-aloud news.
As the stack gets higher, few questions arise: Which of these features are important and engaging to our subscribers and should be developed further? Which should we get rid of to make room for something else?
At Kauppalehti, Finland’s leading financial news outlet, we wanted to give our subscribers a more personalised customer experience by recommending a product feature based on their earlier behaviour that would be most relevant to them. Our team of data scientists and product developers went through usage data of all our different product features and created a data model to predict which product features could be relevant and engaging to each subscriber.
We first tested the model organically: Following the data, subscribers spontaneously started using the features the model recommended to them. The results were promising, so we continued with personalised e-mails to 3,000 subscribers.
We conducted this test in fall 2020, so we do not have long-term results yet. But primary results are encouraging. Compared with a control group, subscriber engagement (measured with FRV) increased with recommendations. Additionally, there were very few negative effects (0.5% unsubscribed from the e-mails). We also learned which product features created the most positive response. A longer test period will show if the engagement stays at a higher level and if it has a real impact on subscriber retention.
These are the first important steps we have taken to understand the real value of the different product features we have. Maybe one day we will have clear answers on what we should keep in our stack and what we should disregard. This evaluation is much easier with features that we see converting new customers, but how do we put a price tag on these engaging features?
Too many times you hear “this podcast” or “that newsletter” is very important to some of the subscribers, and that is why we must keep on producing it. But what is the real value of it? Is it actually that important to those subscribers? Could we even put a monetary value on all these features, so we could estimate their ROI with at least some degree of precision?
These are some of the questions I hope to explore about product development in the coming year.