A report by the Reuters Institute for the Study of Journalism at Oxford reminds us that more than 40% of AI-using news consumers under the age of 55 are looking for summaries of the news. 

It is hardly a surprise, then, that we find news organisations all over the world are looking at ways to summarise their content.

What are the best practices around summarisation? 

• Your LLM should be guided on getting the five Ws in the summary and be prompted to generate fact-based, neutral summaries (so you’re not getting loaded language). An editor should test these for neutrality and accuracy before you make the decision to go live with summaries for your audience.  

• Disclose to your audience that the summaries are AI-generated and they should flag any inaccuracies for you. It could be at the bottom of the article in italics, or it could be in a pop-up/interstitial when they hover their mouse over the summary, for example.  

• The human in the loop is important here. The best practice is that editors cast an eye over the summaries (which are usually at the top of articles) to ensure they are accurate and surface the most important points. They are usually editing the articles as well, so this extra step adds only a few seconds of work. 

• There’s also a handy open-source tool that actually checks summaries against source content to ensure the facts are consistent. (Note: This is not a fact checker; if the facts in the original article are incorrect, it will not catch that.)

• Summaries tend to work well on most news stories, but there are some places where they do not:

• Columns, where an author is making a nuanced case for a particular opinion.

• Breaking news/reverse chronological live blogs. The machine gets confused because the article is not nicely structured to send signals about what is important and what is not. (The Guardian tried this. You can read about it here.)

• Legally contentious stories. The LLM is likely to miss context and certain subtleties associated with facts and could easily get you into trouble by baldly stating selected facts.

• Satire. I am not sure the machine can pick up on that.

• Emotionally charged articles. Ask the editorial team to exercise extra caution with such articles as the LLM has no sensitivity or judgment and can get the tone absolutely wrong in a summary. 

• Very short stories (eg, a news flash of a paragraph or two). The summary doesn’t really add any benefit. 

• It’s also important to frame summarisation as an experiment, test AI-generated summaries on various types and lengths of articles, and then provide editorial with the results to see where it worked and where it didn’t. This is new, rapidly changing technology, and it’s unlikely anyone has all the answers at this point. 

• Agree in advance on why you are running this experiment and how you’re going to measure its success, e.g., changes in user engagement as measured by pageviews per visit or time spent per visit, for example.  

After trying summaries, Norway’s Schibsted discovered article read times actually increased. The Hindustan Times discovered bite-sized news drove higher engagement.

Some news brands have realised their readers are tight on time — and serving them more information in a smaller time frame may prove to be the recipe for success. Certainly that was a key discovery at the Frankfurter Allgemeine Zeitung, where the team decided to test AI-generated summaries as a way of retaining subscribers after learning they were feeling pressed for time.

The FAZ team was pleasantly surprised to see the summaries started driving not only retention but also conversions. It’s a good reminder that content serves many different needs — sometimes readers just need a quick hit to update them on what’s going on in the world.

Meanwhile, Argentina’s Clarín also realised different readers consume and process information in different ways. So they offer news consumers a range of options for receiving their news — as bullet points, as highlights, as data, as FAQ, or as a glossary.

This “improves the quality of the information diet by presenting the main information in doses different from the originals, allowing a person to access a greater quantity and quality of information in less time,” Clarín’s Julian Gallo said.

If you’d like to subscribe to my bi-weekly newsletter, INMA members can do so here.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .