Newsrooms usually want to experiment with new strategies, but they don’t always know the best way. The challenge is knowing how to prioritise, plan, and look back on the data effectively. 

Uma Patel, News Lab lead at Google/Australia and New Zealand, and Gaven Morris, managing director at Bastion Transform at the Bastion Agency, have been working with newsrooms to establish frameworks so they can innovate and run experiments effectively. One thing they’ve learned along the way, they told attendees at the recent INMA Asia/Pacific News Media Summit, is that encouraging innovation requires establishing the right culture around risk.  

“Newsrooms are built on healthy skepticism, which is great when you’re thinking about … holding the people in power to account, but when it comes to thinking about the opportunities, sometimes it works against you,” Patel said.

Use what you’ve got

Drawing journalists in involves using their strengths to the best advantage. For example, newsrooms thrive when there is a vacuum of information, and many journalists get their energy from digging up that information. And of course, journalists are bulldogs with research and are constantly iterating. 

Given all of these strengths, Patel shared some basic steps for newsrooms to experiment.

• Prioritise: Rank opportunities according to size and effort and then test them.

• Be ambitious: Embrace risk and run toward it.

• Use data: Design experiments with measurements in mind, and set benchmarks for success. 

• Make a conscious decision: Avoid “zombie experiments,” which can happen when newsrooms accumulate data but still have no clear decision about what to do with it. Be prepared to make a clear decision to either grow, pivot, or kill the experiment. 

Embrace uncertainty: Start in a space where you don’t have the knowledge yet. 

Coming from a newsroom, Morris said he remembers one thing that always kept editorial staff from being able to experiment effectively was just not having the bandwidth to step outside of their routines in a fast-paced environment. So when helping them incorporate frameworks to try the experiments, it was important to remember the usual rhythms of how newsrooms actually work to allow them to step away and find the headspace to experiment.  

Morris and Patel organised a series of workshops with three very different newsrooms: Business Desk in New Zealand, National Indigenous Times in Australia, and Australian multimedia company Nine. They found all of them tended to be more cautious than they should. To encourage them to reach further and take some risks, the team would advise, “If you don’t think this could fail, then maybe it’s not bold enough.”

Morris coached them to sort ideas into categories: things you would do anyway, things being done elsewhere, and things that are just a little too crazy. Even when focusing on the third category, the proposals seemed modest, but it did help everyone stretch their thinking.

• Business Desk in New Zealand is a startup, so its newsroom was already used to a certain amount of experimenting. The team wanted to get right into AI and research how it could reduce the burden of certain time-consuming daily functions like write-ups based on market and stock reports. The workshops helped them set up a process to get the AI engine to pull out factual information without analysis and made progress quite quickly. The team also looked into whether they could take something very specialised that they were reporting on and expand it to an international audience through a newsletter.

• National Indigenous Times, a small indigenous publication in Australia, is growing fast and is quite ambitious. Its team wanted to explore international connections to its content covering indigenous issues. 

• Nine, is a prominent publisher in Australia that owns many daily newspapers. Its team wanted to be more ambitious about testing the parameters of their paywalls and subscription business and, in particular, to test the appetite of their subscribers to pay for more high-value content behind a paywall. They also wanted to explore video content.

Lessons and recommendations

They are still crunching numbers on the results of the workshops, Morris said, but it is already clear that as ambitious as all three were, he can see one lesson is everyone needs to be even bolder: “We pretty much smashed all the targets that we set.”

Patel also had advice on developing metrics. It's important to start with well-defined measures that can be used as benchmarks for success — for example, quantitative (like pageviews) as well as qualitative (shares, time on page). It’s also important to establish baselines using historical data and to make sure metrics take risk into account. And as important as metrics can be, there are also important but unmeasurable benefits to remember — like the value of establishing a culture where the newsroom is willing to take some risk. 

“Getting newsrooms in the rhythm of doing regular experiments isn’t just something that’s nice to do. It’s really essential,” Morris said.

Another positive cultural change Morris pointed out: The frameworks established an avenue for newsrooms to share more information with each other. 

Preliminary results of the study showed even modest experiments are really paying off, he said. The most dramatic example is at Nine, which saw saw a 300% to 400% improvement in subscriptions once they brought the newsrooms into the discussion. 

The other two newsrooms saw more subtle but also impressive changes. Business Desk’s AI strategy was so successful the company is now talking about adopting the idea at a parent-company level. And Indigenous Times’ relatively modest experiment brought a 20% growth in engagement in international content — much more than what they were expecting. 

The key takeaway, Morris said, is that experiments are paying off, and the more ambitious the better.

Data case studies in experimentation from Google News Initiative

Data experimentation, of course, is adjacent to anything that happens in the newsroom. The Google News Initiative (GNI) has partnered with large and small news publishers across the world to help them better shift to a data-first approach. Nitan Sharma of Google India’s News Partnerships shared three recent case studies that illustrate the wide range of possible scenarios.

South China Morning Post: The 100-year-old publication has successfully shifted from being a traditional print newspaper to a digital-first publication. GNI worked with them to help build a recommendation engine that boosted user engagement. The engine dynamically creates user types based on various data sets, sorting users based on demographic, behavioural, and psychographic data. This allows SCMP to serve up the right content and call to action for each user, and combines the algorithm with editorial curation. 

Click-through rates from the “People Also Read” widget increased six times, and average time on page increased 25%. 

CommonWealth Magazine: GNI worked with the Taiwanese business magazine to develop a new dashboard that increased engagement with active users and drove new membership. The team realised the magazine needed a common membership structure and created a system that segmented users by engagement level, offering different monetisation levels accordingly. 

The shift from content-centric to more user-centric data resulted in a 15% year-over-year increase in membership.

New Zealand Media and Entertainment: The print, digital, and radio company, which owns 30 brands in New Zealand, updated its content management system for all titles to streamline workflows, reduce complexity, and improvement user engagement. This enormous task had to be rolled out slowly and involved training 300 journalists but ultimately allowed the company to publish content more efficiently with less room for error.

The upgrade led to measurable improvements is user engagement including a 5.85% decrease in bounce rate and 7.5% increase in average page-view sesion duration.

Details of these and other case studies are available on the Google News Initiative site.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .