Here’s an interesting multimodal GenAI use case, this one from Axel Springer-owned Politico, in Washington, D.C.  

The political news site often needs to cover live events, such as debates, where many presenters are speaking in quick succession on a range of topics. Politico wants to provide its audience with a live blog so they can keep abreast of interesting developments, particularly on topics that interest them.

“One of the challenges we’ve always had to solve is: What do we do in the moment of a live event, a debate, or a speech?” said Andrew Briz, editorial director of newsroom engineering

But existing live chat tools provide a “disjointed” experience, particularly if a reader comes in after the event has started and is trying to keep up with the live blog while scrolling through what has already been said, Briz pointed out. He also wanted to give readers the ability to filter the conversation to focus on particular topics.

“Now, with AI, it is realistic to do a constantly updated, up-to-the-moment summary,” which can be categorised by topic so readers can quickly find information relevant to their interests — and so journalists can focus on analysing the content rather than simply transcribing or reporting it, he said.

The tool ingests an audio feed, which could be either pure speech or video, transcribes it, cuts it up, and identifies which speakers are speaking (editors need to give it that information once). The editors create four topics they think are important. The tool undertakes a retrieval augmented generation (RAG) search on those topics, finds the parts of the transcript that apply to that topic, and creates a summary.

The next step is to improve the tool so the quality of the content generated is strong and clear. At the moment, an editor needs to go over it carefully to ensure there is no gibberish published. 

How does the audience feel about this product? Politico added “thumbs up” and “thumbs down” buttons and asked its readers: Is this helpful?

“My thought originally was that these were tied to summaries themselves, that people will ‘thumbs up’ good summaries and ‘thumbs down’ bad summaries, and then we’d be able to see the good ones from the bad ones without our having to go in and rank them all,” Briz said. 

“What happened instead was whatever was at the front of the line got the most engagement, which makes sense — most people saw those buttons and they didn’t necessarily click to the other summaries. But what was surprising was almost every summary had exactly a 50-50 thumbs-up to thumbs-down ratio, whether it was the first one, which had tons of engagement, or one of the later ones that had much less engagement. 

“My read on that is that they are reacting to the technology itself, not to the actual question: Was this helpful. I think we’re suffering a little bit here from a backlash of some people really don’t like this technology and they’re just going to hit that no button every time. I was expecting to see some sort of distribution and it’s non-existent.”

If you’d like to subscribe to my bi-weekly newsletter, INMA members can do so here.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .