Do you think you could get more audience engagement from your lineups? That’s the question we recently asked ourselves at Canadian Broadcasting Corporation.

Across the whole company, we publish 300 to 400 pieces of content a day into roughly 40 different lineups. We have a lineup for our home page, lineups for our apps, and topic-specific lineups like sports, health, and business.

Although there’s currently no personalisation on these lineups, they seemed to perform pretty well. Bounce rates aren’t terribly high, the click-through rates seemed reasonable. But we still wondered if we could get more engagement out of them.

“We have a trove of high-quality content and wondered if we could do a better job of playing matchmaker between the audience and the content,” said Pras Rajagopalan, the news executive producer who led this testing.

The experiment

We set up a test with the objective of increasing engagement, measured by time spent. We wanted to increase the conversion from landing on a lineup to consuming a story. We decided to test different approaches using an A/B test on our front page lineup in our news app.

Mathew Martin, the product manager for our news app, said: “We looked at app user behaviour, how frequently they opened the app, and patterns throughout the day, as well as user feedback. Based on those, we defined some tactics to better meet those needs through curation.”

The tactics included dramatically, increasing how often we published content into the lineups to being much more sensitive to publishing relevant content to particular time zones and republishing content on weekends.

The result of implementing these tactics was that the number of stories in the lineup during the test period went up by 270%. We also greatly increased the proportion of local content, which went up by 670%.

The results

Story loads, meaning the number of instances a user loaded a story from the lineup, went up by 38.4% relative to the control in the A/B test. In addition, video streaming increased by 46%.

Because these tactics were applied to one of our most popular lineups, the results have implications for the whole company, as they can also be applied to other lineups. In addition, increasing engagement from the lineups has an obvious impact on the revenue we generate from our stories.

“When we ran A/B tests on our standard curation tactics, we saw greater than a 30% lift in content opens. We have a correlated lift in time spent. It wasn’t just clickbait — in and out. It was meeting a user need because by the time spent went up. We can confidently say they are getting value out of that consumption,” Martin said.

We have proven from this testing that it’s possible to get substantially higher engagement from lineups, but it may take a lot of work. “To be able to do this lineup testing, there was a lot of manual lift,” said Joanna Vito, the product owner for the news app.

The next step

The next phase of this project is to look at how we can automate and personalise some of this work. How can we get more engagement from our lineups without having to do it all manually? What are the pros and cons of automating lineup management tasks? How can we introduce automation and personalisation while still fulfilling our mandate of keeping all Canadians informed about the news they need to know?

These are questions we’re tackling next.

“There are a lot of implications about how to personalise content for individuals. How consistent do you need to be in presented content to audiences? What news should you be showing to all users?” Vito said. “It’s the balance of mandate and curation. What is the balance that everyone feels is appropriate?”

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .