Experimentation plays an important role in how news media companies test out new hypotheses, identify what works, and learn more about reader behaviour.

During INMA Media Subscriptions Town Hall, members learned how The Irish Times, Croatia’s Hanza Media, and Portugal’s Observador used experimentation to grow their audiences as well as change their internal culture.

The companies participated in the third edition of the GNI Subscriptions Academy for Europe. The recording and presentations can be found here and an overview of the Town Hall can be found here.

The Irish Times benefits from experimentation culture

Cliona Mooney, subscriptions and reader insights director for The Irish Times, said embracing a mindset of experimentation accelerated learning for the company and allowed it to prioritise its projects. But it has other benefits as well: It also saves money, promotes reader-centricity, and builds a data-driven culture.

“Before we started on the programme, there was some experimentation done in the organisation, but it would’ve been more siloed and ad hoc,” she explained. “We wouldn’t have had that structured approach so we probably didn’t end experiments, really.”

Though it had the best intentions, “we didn’t really have an experiment culture in that sense of communicating outside various departments as to how experiments had run.”

And that is a crucial point many companies miss. Once it understood how to define the methodology and create a structured, templated approach with open communication channels between departments, The Irish Times saw better results and was able to develop a 12-month action plan of hypotheses to experiment with.

Now, it has a collaborative and cross-functional approach that allows all departments to be involved and provides a better overview of what is working — and why.

“Experimentation helps to learn what works and what doesn’t work with our audience,” Mooney said. “The structured approach really helps [give] that clarity on being really clear on what we are testing and what we are not testing.”

And it has taught The Irish Times not to be afraid to fail, instead learning to use so-called failures as a learning experience: “If something doesn’t work out, we can tweak it and try a different iteration of it,” she said. “That has been really key for us.”

Hanza Media creates new processes from experimentation

At Hanza Media, the programme changed how it looked at experimentation and created new processes, said Stipa Grubišić, chief digital officer. He said it made the company more aware of what it was testing for and, as a result, the ideas that people brought to the table were much more thought-out.

“The ideas people were bringing to the table were not whole,” he said. “It was just, ‘I thought of this, I envisioned that.’” But oftentimes, they didn’t know what kind of results they should expect.

Grubišić said he finds the collaborative way of thinking more encouraging: “So you call the data guy or somebody from editorial or somebody and say, ‘Look, I have this idea, does this make any sense?’ Which is a process in which people learn.”

The company isn’t data-driven, but it has become more heavily data-informed as it has embraced experimentation, he said: “People can still come with ideas, but you can structure the conversation with them to make something out of this idea. Some ideas don’t sound great to me, but if half of the team says we really think we should do this and … we have written it down, this is what you can expect — I cannot say no to that.”

Observador is more competitive with focused experimentation

Leo Xavier, director of product management at Observador, said although the 8-year-old publication already fostered a climate of experimentation, it lacked “the framework and methodology to do it in a very consistent and strategically targeted way.”

News media companies are inherently filled with creative, passionate people bursting with ideas, but what Xavier wanted to do was “make sure that those ideas contribute in a specific way to what we want to achieve.”

Experimentation allowed it to better focus on its strategic vision: “I’m sure we’re not the only newspaper that feels that we could do with more talent, we could do with more digital, highly specialised skills. And when you don’t have enough resources, you need to prioritise and focus and make sure that you’re picking up on the right things.”

By creating a list of its possible experiments and prioritising what could be either quick wins or big opportunities, Observador has become more competitive. Xavier noted it isn’t just other newspapers Observador is competing against; it is digital companies like Netflix, HBO, and other platforms that are vying for the same monthly subscription fee.

“I think we’re much closer to becoming competitors to them now because in terms of experimentation, a data-driven approach, a user-centric approach, these companies are best of the breed,” he said. “And I think this gap has to be reduced between newspapers and non-newspaper content producers when it comes to subscriptions.”

Experimentation can help close that gap, particularly if companies will accept that some ideas will fail and use the learnings from those failures: “We need to accept that some ideas will fails, others won’t, and all ideas are welcome, but many of them won’t work. If you don’t accept the failure of it, then I think it won’t live on.”

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .