Measuring the sentiment of readers gives publishers the “why” as well as the “what”
Product Initiative Newsletter Blog | 11 July 2023
Hi there.
The U.S. has been consumed with Threads vs. Twitter, and I have to admit that for once it seems a pleasant thing that a Big Tech company has ripped off someone else’s design. Kudos to Meta for pulling off an exceptional launch with the largest sign-up of any single new app to date. I’m on there as @jodiehop so please connect!
I wasn’t planning on doing a breakdown of the app because it’s been done so much elsewhere. But I did want to note that Adam Moserri, who heads this and Instagram, has specifically expressed that their hope is the app isn’t used for hard news and politics, but more to connect people around common interests such as art, music, and sports. Of course, one could argue news and politics is everyone’s common interest, but the point here is this is not deliberately going to be a traffic driver for this industry.
Back to today’s newsletter, where we’re looking at patterns from two extreme points of view. The first is getting a gauge on users sentiment analysis to take the temperature of a nation. This is what Stuff in New Zealand has been working on, so we take a look at their MVP in action and what they have learnt to date.
The second is a brief look into dark patterns, a phrase that has come to national attention here in the U.S. as the FTC starts investigating Amazon. The results may have a knock on effect for media, so I wanted to take an early look at what dark patterns are and how they are being used to trick customers.
As always, please drop me a note if there is something in here that resonates or something you’d like to discuss. I’m at Jodie.hopperton@INMA.org.
Thanks, Jodie
Getting the sentiment analysis of a nation: an MVP in action
While I was nosing around Stuff’s Web site looking at how audio is built in (more on that here), I noticed a sentiment tracker. When I asked CPO Ben Haywood about it, he told me that “when combined with the article metadata, the data captured from the survey allows us to understand audience sentiment on issues over time.”
I was intrigued, so I delved further.
The initiative is driven by the data team, and so I spoke to Dina Hay, chief data and insights officer, and Amanda Lane, head of research and insight, to learn more about the thinking behind it, what they have learnt, and how they want to expand it in future.
To give some context, this is the sentiment tracker that currently appears on the site:
The Stuff team noticed something similar at Rappler in the Philippines (run by the incredible Maria Ressa). Once you have taken part on their site, you see the results from other people (in the same way Instagram polls run).
If there is enough data, you can track sentiments towards issues, people, and brands at any given point, or how it changes over time. Stuff wondered how it could be used in New Zealand, so the team has developed an MVP and has done a lot of testing over the past few months.
What I hadn’t seen, because I hadn’t originally clicked, is this:
They are gathering quantitative and qualitative data. Anyone who knows me knows that is music to my ears ;) It allows them to go even deeper with the “why” as well as the “what.”
Their sentiment tracker is now beyond an MVP and is shown to most readers with a couple of exceptions:
- Editorial can exclude it if they think the article is inappropriate for this purpose.
- It is limited to scroll speed and depth so it only shows to people who have read the article.
So why do they think this is worth the investment?
First and foremost, Stuff considers this a B2B tool. It’s a revenue driver. They have their finger on the pulse of the sentiments of the nation, and they can monetise that with businesses, brands, industry bodies, public figures, and probably more.
The team is working out the boundaries of this. I got excited by the potential to use this to understand user needs. For example, if you look at The Atlantic user needs (article here), something that came up was “give me a break.” A news organisation could therefore use this to see if they are publishing too many hard news stories and need to add some softer, more positive sentiment stories.
With advanced sequencing, the next read could directly relate to the sentiment a reader has just registered. Dina and Amanda were quick to say they are ironing out the framework for this to be used. It can’t be a free for all access internally until they understand the possible downsides (just think about how Facebook can skew information by using sentiment).
President Sinead Boucher told me: “We absolutely see the power of this as a tool to improve our content and experiences for our audience, as much as for the potential revenue stream. One of our strategic goals is to have the deepest and richest knowledge of New Zealanders and to use that to develop great content and products for them.
“Stuff’s vision is to be the most trusted organisation in New Zealand. We know from our external research that part of what builds trust is that users feel they are understood, listened to, and respected. So that is very much at the front of our minds here, too, both with how we collect and use the data, but also in how we use it to enrich what we produce.”
Sentiment analysis is in the early stages, but it’s an interesting project. They get 14k responses per day, which is a statistically meaningful data point for a country of five million people. Yes sentiment tracking has been done before, notably by BuzzFeed, but I don’t remember the results (this may be my bad, not theirs). But when posted on all news, and especially when combined with other data, the results will not just be interesting. They will be actionable and monetisable.
One question I learnt from my colleague Ariane Bernard, lead for the INMA Smart Data Initiative, is that data gathering of this kind can often be at the polar opposites. For example, I only post two types of reviews on Yelp: for places/services I love and those I hate. There is no middle ground. Amanda told me they have consistently seen a range of emotions during the trial phases, suggesting that this wasn’t the case here.
They have seen a number of advantages launching as an MVP, in many of the ways you would expect: seeing if and how people use it and enabling UX tweaks before full rollout. But something else significant happened. The tool was spotted by a professor at a local university who contacted them to discuss it. He was so curious and it fit so closely with his work that he became an adviser.
Through his input, they have been able to work on combining sentiment to enlarge the data pool from six emotions to 36. In the future, users will be able to choose two sentiments, which gives more complexity, e.g. “concerned” and “angry” could equal “anxiety.” When launched, this will give even more data points to use.
There is much that will be rolled out in the next 12 to 18 months to develop this tool further. We’ll be following to see what other results and surprises it brings.
Date for the diary: August 9, free INMA member Webinar on using AI voice to narrate articles
Have you noticed audio is cropping up more and more as a format that’s integrated into main news products? Not just the NYT audio app. You may remember a post on audio and how Schibsted has seamlessly integrated narrated articles into the main app.
In this free Webinar session, we’ll hear from Lena Beate Hamborg Pedersen, product manager of products and technology at Schibsted, on the origins of this and how it has developed and has since been used. This is a story everyone can learn from and use today.
You can register for the event here.
The end to dark patterns: what you need to know (and possibly do)
The Federal Trade Commission (FTC) in the U.S. is now investigating Amazon Prime, saying it “used manipulative, coercive, or deceptive user-interface designs known as ‘dark patterns’ to trick consumers into enrolling in automatically renewing Prime subscriptions” and also kept customers from cancelling.
Why does this matter for you? Because the design tools that are used for subscriptions are under investigation, and a severe result here may restrict the ways that you use UX for sign-up, onboarding, and cancelling, in the U.S. at least.
Don’t worry just yet as this isn’t imminent, but I do think it warrants a moment’s thought.
What exactly is a dark pattern?
It’s defined as using “design practices that trick or manipulate users into making choices they would not otherwise have made and that may cause harm.” And what may cause harm on our sites you may ask? People parting with money or stopping their ability to stop paying with money. It may also refer to content that is disguised advertising. Effectively, it’s using UX to lead consumers down a certain path, which may either not be what they want to do or something they don’t understand.
It’s a balance for us because we want to use tools that encourage people to sign up, but we also have to be clear about what they are signing up for. Trial pricing moving to standard subscription pricing is likely to be a target, as well as people’s ability to cancel, at least in the Amazon case.
Here are the three main elements of the dark patterns policy:
Disclose clearly and conspicuously all material terms of the product or service, including how much it costs, deadlines by which the consumer must act to stop further charges, the amount and frequency of such charges, how to cancel, and information about the product or service itself to stop consumers from being deceived about the characteristics of the product or service. The statement provides detail on what clear and conspicuous means, particularly noting that the information must be provided up front when the consumer first sees the offer and generally as prominent as the offer itself.
Obtain the consumer’s express informed consent before charging them for a product or services. This includes obtaining the consumer’s acceptance of the negative option feature separately from other portions of the entire transaction, not including information that interferes with, detracts from, contradicts, or otherwise undermines the consumer’s ability to provide their express informed consent.
Provide easy and simple cancellation to the consumer. Marketers should provide cancellation mechanisms that are at least as easy to use as the method the consumer used to buy the product or service in the first place.
Europe’s Digital Services Act (DSA) is bringing a similar rule into effect in February 2024, stating: “Providers of online platforms shall not design, organise, or operate their online interfaces in a way that deceives or manipulates the recipients of their service or in a way that otherwise materially distorts or impairs the ability of the recipients of their service to make free and informed decisions.”
In some ways, this is more strict. Here are three of the points they make:
Presenting choices in a non-neutral manner — such as giving more prominence to certain choices through visual, auditory, or other components — when asking the recipient of the service for a decision.
Making the procedure of cancelling a service significantly more cumbersome than signing up to it.
Making certain choices more difficult or time consuming than others.
At best, this is to help consumers navigate online payments and subscriptions. At worst, regulation may restrict the way we advertise and move people through the funnel, onboarding, and cancellation processes. If I was running a product team right now, I would check to see which parts of the products need more scrutiny or adaption before these rules come into play.
About this newsletter
Today’s newsletter is written by Jodie Hopperton, based in Los Angeles and lead for the INMA Product Initiative. Jodie will share research, case studies, and thought leadership on the topic of global news media product.
This newsletter is a public face of the Product Initiative by INMA, outlined here. E-mail Jodie at jodie.hopperton@inma.org with thoughts, suggestions, and questions. Sign up to our Slack channel.