The Wall Street Journal (WSJ) strives to be the world’s most trusted source of journalism to help people make decisions. As members of its engagement team, our goal is to ensure we deliver WSJ’s trusted journalism to its audience of decision makers in the most engaging and relevant way possible.

When the engagement team was first created, we sought new ways to communicate important messages to our existing members that went beyond standard marketing channels. In collaboration with the product team, we created on-site ad placements on WSJ.com to promote member benefits and “healthy habits” we identified as actions that can increase engagement and product usage to prevent churn.

With years of testing and advancements in technical capabilities, our on-site member messages have undergone a huge transformation. We now have the capability to encourage members to take thousands of relevant actions each month. The following key insights that have helped us to optimise and make the most of our on-site real estate.

Focus on improving the overall design

The engagement team has ownership of three on-site ad slots. This means we are able to populate these placements with content or products, and target specific users with personalised messages. Our ad slots are as follows:

• Candy bars: These appear as skinny pop-ups at the bottom of the screen.
• Right-hand side tiles: These appear under a user’s name in the top right corner of the site.
• Membership tiles: These appear as a small box three quarters down the page on the homepage and article pages.

After a few years of gathering data on the performance of these placements, we shifted our focus to testing and optimising their overall designs to make them work harder for us. We learned the look and feel of our on-site ad placements is key. By making small, incremental changes to the design, we can deliver valuable increases in an ad’s click-through rate (CTR).

With our candy bar placement, for example, we determined that a plain-text version often has a higher CTR than a version with an image and other, more visual, design elements. In the example below, we promoted the launch of a new WSJ podcast, Bad Bets, with both a designed candy bar and a plain-text candy bar. The results spoke for themselves: The plain text version’s CTR was 33% higher than the more designed version.

Throughout our testing, we have discovered that a simple, clean design is often more engaging to users. One working hypothesis for this is that overly designed placements can start to closely resemble paid advertisements, which might result in users unconsciously skipping over them.

Make the most of the space you have

Once we adjusted our ad creative to present a simpler design format, we then experimented to see how we could make the most of the real estate on each ad slot. One test we ran was adding a secondary call to action (CTA) to our membership tiles and candy bars. We wanted to leverage the Hobson’s Choice +1 effect, in which giving users a choice between two options increases overall engagement.

In the following example, we tested promoting a specific e-mail challenge (a series of weekly e-mails users can start at any time), plus gave members the chance to learn more about all of our e-mail challenges.

We observed that membership tiles featuring the dual CTA variation had a 70% click rate uplift compared to our control with just one CTA. We’re also seeing similar results on a test of dual CTAs on our candy bar placement as well.

Based on our learnings, we recommend continued testing to understand the best way to leverage on-site placements. Do not become complacent. There will always be elements that can be further optimised, whether it’s a design change or simply adding another CTA.

Automate what you know works

Now that we’ve uncovered what is performing well with our on-site placements, our focus has shifted to reaching scale through automation.

For example, we know from our previous research that the first 100 days in a subscription are crucial for member retention, so we created a special on-site experience for new members. By leveraging automation technology, we are able to show specific on-site messages to new members every day that encourage the adoption of healthy habits to prevent churn.

We are also leveraging dynamic capabilities to set up automated candy bar campaigns that pull in content from feeds. We can set up a candy bar once and then have it show the latest content going forward, without our intervention. We use this a variety of ways, including:

• Promoting a trending episode of a podcast.
• Highlighting the latest column from a weekly series.
• Making personalised article recommendations based on a member’s reading habits.

Through automating our routine campaigns, we are delivering more engaging and personalised experiences, while also giving the team time to improve the ad placements overall. Since we can set the automatic campaign once and let it continue to run, we are also able to create super specific segments to target — something we would not have had the resources to do every week. In just the first two months since these tactics were implemented, we have seen a 200% increase in clicks.

Reach members who are most likely to engage

Once we successfully automated the content for some placements, we then looked into how we could automate the targeting too. Instead of simply relying on our best instincts of who would be most likely to engage with certain campaigns, we partnered with our operations and data teams to create “look-a-like models” that identify people who look and act just like our target audiences.

For example, instead of promoting our recipes to members we think would be interested, we are able to target it to members who have similar behaviours as members who already engage with our recipe content.

These teams also created “channel preference models,” which allows us to establish where to reach our members on an individual basis to increase their likelihood of engaging with our message. There are some members who might just never engage with on-site placements, even with all our work to optimise them. Instead, we need to reach them through other channels such as via e-mail, in-app messaging, or paid media.

By leveraging these two models, our ultimate goal is to scale our audience size and appeal to members where they are most likely to engage with us.

Get started on your own on-site journey

Our advice to teams working on similar challenges is to first focus on optimisation and ensure your on-site components are the best they can be from a look-and-feel perspective. Once you know what works, you can focus on automation to move away from a manual, one-size-fits-all approach and start leveraging machine learning to drive more meaningful and scalable engagement.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .