Greetings, everyone! I just got back from the INMA Roundtable at Vail, where I had the privilege of spending three days in discussions with 50 leaders of member companies. 

As they prepare for 2025, they gathered to discuss future-proofing their businesses.

Today’s newsletter looks at the AI-specific angles of our conversations, as well as at some great insights into building chat products from two innovative news brands.

Sonali

GenAI and change management: What does it take?

We asked the leaders at Vail to outline what they were concerned about going into the session. They said:

• Structural and organisational change needed to benefit from AI.

• Product and audience development in the AI transformation.

• AI for efficiencies.

• How to prioritise AI initiatives.

• Grabbing the AI opportunity.

We also asked them after the conference what they were going to do upon returning to work, and they said:

• Prioritise AI plans.

• Put AI in the heart of the transformation.

• Use AI to do newsroom activities that free up journalists’ time.

• Take a more structured approach towards AI.

• Hyperlocal and personalisation AI are feasible today.

• Rethink our AI structure, resource alignment.

• AI: Start with efficiencies and move towards growth as it becomes more comfortable.

• AI: Raise awareness, expand basic uses across team, leverage champions.

• Consumers will accept AI content.

• Integration of AI into different projects.

• Integration of AI into CMS.

• Take the LLMs’ money/make deals with LLMs if you can.

Some other key points that emerged from our discussions:

• Don’t be hung up on ROI — it stifles innovation. Some CEOs pointed towards the tension between the CFO and the team running experiments. Others pointed out that seeking returns too early can kill any spirit of innovation, especially when dealing with a technology as new as generative AI, where trial and error has to be factored in.  

• Everyone is building GenAI tools, but not everyone in the news organisation is excited about using these tools. A few participants at Vail mentioned a 25% to 30% adoption rate.

• A show of hands suggested about half of the people in the room said they were tracking the adoption of AI tools. About half are creating tools for their journalists and are not even measuring whether they are using them or not.

• Change management takes some effort. An estimate by McKinsey & Company shows companies need to spend US$3 on change management for every dollar spent on a GenAI application. 

On the latter point, there was a robust discussion about whether this number could possibly be accurate. Surely it was too high?

Here is what McKinsey says about that: “Our experience has shown that a good rule of thumb for managing GenAI costs is that for every US$1 spent on developing a model, you need to spend about US$3 for change management. (By way of comparison, for digital solutions, the ratio has tended to be closer to US$1 for development to US$1 for change management.)” 

What does change management look like? It entails several efforts, such as:

• Training programmes for staff to use new AI tools.

• Restructuring of workflows to incorporate AI-assisted processes.

• Development of new policies and guidelines for AI use.

• Addressing ethical concerns and ensuring responsible AI use.

• Adapting business strategies to leverage AI capabilities.

Here are some examples of the resources media companies are putting into GenAI change management:

• Schibsted’s Aftonbladet has trained 260 journalists in prompt engineering and has seven full-time positions on its AI hub.

• Tamedia has onboarded 1,300 employees by going on a roadshow to specific teams and training them, while also writing a newsletter and setting up a prompt library.

• Mediahuis has set up an AI task force within each of its newsrooms and has a central coordinator who trains and supports the adoption of AI tools with each newsroom lead. It has 4,500 employees at more than 30 news brands.

Still not convinced? Disregard cautionary tales at your own peril of companies not thinking through GenAI implementation, such as this one about employees accessing sensitive salary information through Copilot summaries.

GenAI chat products: insights from two prominent publications

Many of the 125-ish INMA members whom I have heard from or spoken to over the past nine months are working on creating chat products. In fact, we had 188 participants registered for a Webinar on chat products on September 4, so we asked them a couple of questions to get some idea of what people were thinking. (Feel free to watch the replay — come for the chatbots, stay for the Beethoven and the Lord of the Rings reference.)

The first was whether they have a chat product in the works already (admittedly, not the best question since presumably all viewers of a Webinar on chat products would be at least somewhat interested in the topic). A quarter of the respondents had already built one, one-fifth had one in the pipeline, while about 40% were considering creating one.

We also asked what they were worried about.

It looks like the biggest concerns over chat products centre on accuracy and profitability, which are concerns that we have discussed here before. But it was also interesting to see that some respondents flagged utility as a question mark: Is this actually a useful product that serves a need that the user has?

In fact, rapid prototyping — allowing you to see if the product solves the problem you think you have, learn quickly and fail quickly if needed — has emerged as one of the best practices around building a chat product. 

Here are some other useful insights from our two guests, The Washington Post Senior Editor for AI Strategy and Innovation Phoebe Connelly and Aftonbladet Deputy Managing Editor Martin Schori:

• Scope it narrowly. Ideally, on a topic that is not frequently and rapidly overtaken by events so the information it provides remains current.

• Be transparent. Tell your audience what you are thinking and doing. Both Aftonbladet and The Washington Post link their bots to a page that explains how the bot comes up with answers and why they built it.

• Train it on your own content for accuracy and use a RAG model. The object is to provide the curious news consumer with an entry point into a subject you have covered with both breadth and depth, and to help them quickly find what is relevant to them.

• Many users understand AI can make mistakes. Make it clear this is experimental technology and encourage readers to read the content that it links to.

• Users don’t mind AI. “If you ask them, they will say, oh, I don’t want anything that is AI-generated,” Schori said. But: “As long as the information provided is relevant and adds some value, people don’t really care.”

• Teach the bot to say it does not have the answer rather than letting it hallucinate. This does not come naturally to us in the news industry because we are uncomfortable telling readers we do not always have the answers.

• Use the questions readers are asking as data to shape your coverage of the topic. It is a rare glimpse inside their mind to see what is truly relevant to them on a subject.

• Offer the reader some prompts or sample questions to get them started, in addition to offering them a space to type in their own questions. Offer further prompts once those questions have been answered to keep them engaged and to keep showing them value.

• It does take time and work to build a high-quality chat product. (ICYMI: Mike Pletch from Canada’s Globe and Mail delivered a detailed master class session on this in May.) “Just because it’s published doesn’t mean it’s done. A lot of time needs to be spent checking all the questions, going through audience feedback, and adding more data,” as Schori said. “I think it was more work than we thought.”

• Use it to gather first-party data. You can use the chat product to drive registrations. Aftonbladet saw a 40% conversion rate.

• Build an LLM-neutral system rather than being rigidly locked into one.

• Put a daily ceiling on the number of questions that a user can ask if you are worried about cost. Also: Aftonbladet’s bot cost about €1,100 to run over a month.

• The most consistent feedback The Washington Post got was that readers loved the short answers. “Our audience is always asking for brevity,” Connelly said.

• The biggest challenge? Dealing with humans — getting stakeholder buy-in and answering broader questions about the role of AI in news rather than specifically about the product.

Worthwhile links

• GenAI and reporting: Journalists in Venezuela are using AI avatars to avoid being identified and arrested.

• GenAI and inventory management: “Without the use of generative AI, this work would have required nearly 100 times the current head count to complete in the same amount of time,” says Walmart.

• GenAI and model collapse: When generative AI is trained on its own content, its output drifts away from reality.

• GenAI and hallucinations: Francis Ford Coppola’s Megalopolis is a victim.  

• GenAI and gender: Why do so many more men than women use ChatGPT?

• GenAI assistants: Looks like Perplexity is gaining ground on ChatGPT in terms of visits — and is already ahead in terms of engagement.

• GenAI and revenue: Perplexity will start running ads on its chat product in the fourth quarter. Some publishers will benefit from this.

• GenAI and customer-service: Klarna aims to halve workforce with AI-driven gains.

• AI and advertising: Meta’s advertising business is more efficient than ever thanks to AI.

• GenAI and logic: OpenAI is working on improving logical reasoning.

A non-AI diversion

No, really: The famously satirical Onion launches a print edition.

About this newsletter

Today’s newsletter is written by Sonali Verma, based in Toronto, and lead for the INMA Generative AI Initiative. Sonali will share research, case studies, and thought leadership on the topic of generative AI and how it relates to all areas of news media.

This newsletter is a public face of the Generative AI Initiative by INMA, outlined here. E-mail Sonali at sonali.verma@inma.org or connect with her on INMA’s Slack channel with thoughts, suggestions, and questions.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .