Hi there.

I am en route to our INMA CEO Roundtable at Vail in Colorado. It’s a closed-door event held under Chatham House rules, so I’ll report back on some of the broad themes discussed. But in the meantime, I wanted to share some excellent, practical advice I have received recently. 

The first is on AI generated summaries: what to cover, how long they should be, and where to place them. The second is making sure we think about the knock on implications to newsrooms before we take our tools to them.

Drop me a note if there is anything you’d like to see in this newsletter — whether it’s on a particular subject or a feature you’d like to see. I’m at Jodie.hopperton@INMA.org.

All the best, Jodie

Everything you need to know about creating summaries

I recently spoke to Ed Schrager, who has spent the last few years building Summari, a company that creates automatic summaries from content. In a fast talking 45 minutes with me firing questions at him, he kindly shared his top learnings from his years of looking into this. Here are some exceptionally practical helpful insights and best practices in creating summaries. 

A couple of things to set the scene ...

There are two main types of summaries: 

• Extraction, where a part of the text is taken and promoted at the top of the page (easier, but rarely gives an accurate picture of the whole).

• Abstraction, which conveys the meaning of the entire piece using different words and phrasing.  

The challenge of the latter, which is what we are focusing on today, is that you are dependent on large language models understanding the text and synthesizing an accurate summary of that text correctly and — as we have seen from generative AI — that may mean hallucinations. 

So lesson No. 1, which I repeat below, is that there should always be an editor. Ed points out to me that it’s likely a good strategy to check work before publishing, and this applies to AI as well. AI is a copilot. A fairy on your shoulder helping you. It shouldn’t be viewed as a replacement, more so as an enhancement to your workflow. It is not a full-fledged tool that can do everything without some oversight. At least not yet. 

The other thing to think about is that there is always going to be a slight tension with summaries because of the amount of information you want to convey vs. the desire to be as concise as possible. Bear this in mind as you embark on this journey to find the right tension between the two for your audience.

With these things is mind, here are some best practices: 

• Whatever you create must be accurate: You have to give people the most important information as concisely as possible. People can be lazy and just want the highlights.
• Entice people to go deeper: Think of a summary as a movie trailer. It can help readers decide if the full piece of content is worth their time. In many instances, time on page increases 20+% for people that read the summary vs. those that didn’t. 
• Always use bullets (like these!): Ed told me that if you put the same five points in a paragraph vs. bullets, it feels like reading a mini article and is overlooked and less impactful or useful. Bullets are easier to read for the human eye and therefore easier to digest. For standard news articles, use three to five bullets, approx 120 words, and think about the formatting so a human can comprehend what’s going on as fast as possible. Summary length should be between 8%-12% of the total length of the source document — 8% if it's a long piece, 12% if it's shorter.
• Make it clear that it is a summary: Ed praised CNBC’s use of summaries: clearly marked as ‘key points,’ below the headline, above the image. Other examples I have seen aren’t explicitly marked and in some cases look like a sub header rather than a summary.

• Start with context: It’s wise to give readers an introduction or context into what they’re reading otherwise they will feel lost. That should always be your first bullet unless the headline more than explains the article.
• Summarise everything unless it’s less than 250 words: There isn’t a downside to having the summary (apart from time spent producing it) as this seems to appeal to all age groups and demographics. 
• Consider using across mediums other than text: A podcast can be summarised with a few bullets of text. See note above referencing a movie trailer. An effective summary can encourage a reader to view a video and that can have an impact on pre-roll ad impressions.

Product, AI, and the newsroom

I just had a conversation with an AI specialist in a news organisation which reminded me of some of the age old problems of product and newsrooms working together. It’s something we need to be considerate about. And I believe that if we think about the communication and org structures ahead of time, we can have a smoother landing. 

Product people develop new things. We know from the upfront research we’ve done that users will appreciate it and we’re excited to get it into the market. 

But there is a kicker: We’re dependent on newsroom adoption. And with many of the examples of AI I am seeing, we’re asking newsrooms to do more, not less — especially as we recognise the need for “human in the loop” as more examples of AI hallucination come to light. 

Two of the most-used examples are headline generators and summaries. Let’s take each one from a product and a newsroom point of view. 

We can use AI to suggest different headlines. Maybe we have different headlines for different platforms as afterall, they are different audiences in different frames of minds. Excellent to meet user needs, right? But this also means an extra step of time and effort for a journalist to review, and possibly correct, new headlines. 

The same goes for summaries. As I outlined above, so much work can be done automatically to create excellent summaries. But would you send it out without some kind of human check? Unlikely. Again, a journalist will have to read through and make corrections.

As we develop AI tools, we need to also figure out how to bring them into workflows. 

Is it reasonable to expect a journalist to remember to log in to a different system/browser to generate a new headline and then port that back into a CMS? Maybe to start with. But if we want to get mass adoption, we need to take these tools to where our colleagues are already working.

About this newsletter 

Today’s newsletter is written by Jodie Hopperton, based in Los Angeles and lead for the INMA Product Initiative. Jodie will share research, case studies, and thought leadership on the topic of global news media product.

This newsletter is a public face of the Product Initiative by INMA, outlined here. E-mail Jodie at jodie.hopperton@inma.org with thoughts, suggestions, and questions. Sign up to our Slack channel.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .