Hi everyone.

Our little break from generative AI was pretty short indeed — because we’re back on it this week. 

I had three tentpole topics — experimentation, team expansion, and generative AI — for the Smart Data Initiative this year,  but one of them sure is being greedy. 

That said, we are going to mix things up a little and take this to 33,000 feet and talk about the context of this AI — the journey we’re on. This is Part 1, and there will be a Part 2 coming soon.

As always, find me in my inbox (ariane.bernard@inma.org) if you’ve got a great case you want to discuss (or maybe want to speak!). I’d love to meet you.

All my best, Ariane

We should be very aware of generative AI’s relative youth as we consider how to bring it in our organisations

Generative AI asks several ethical questions, and they roughly fall into two groups:

• Questions connected with how generative AI works, how it is built, what goes into it.

• Questions connected with what we do with it and how we understand authorship in this context.

Generative AI frameworks are by their nature built on deep-learning methods and specifically neural networks (not all AI is built on deep learning). 

There is an origin story for us to consider in how deep learning functions. This is crucial for anyone who would care about understanding the values of a generative AI system, but specifically for us in news media. Because if we’re going to use generative AI to create knowledge, we need to be able to appreciate the specific vantage point, the weaknesses and strength of the tools we are using to contribute to our central mission to create knowledge for society.

Think of it as taking stock of the context for the person you are interviewing. Nobody is perfectly balanced on everything, nobody knows everything, and never gets stuff wrong. Still, you can quote a politician — but  you have to have some appreciation for their blind spots or goals. And you have to do some work on your end to fact check, augment, and sidebar the material you are given.

So, this is the context of generative AI: Generative AI has a statistical understanding of our languages, which has been gained by — to put it in a very simplistic way — reading a sizable chunk of the Internet. Reading the Internet is both the source of a generative AI’s understanding of how the world speaks (and therefore how the AI will speak as well) but also the source of its knowledge. 

On the other hand, when we say “Artificial Intelligence,” we cannot lose track of some of the significant, hard limits of what the term truly means: Artificial Intelligence is only statistics. AI writ large has enormous blind spots that make the very label of “intelligence” a very debatable one:

• AI doesn’t understand causality.

• Doesn’t understand the intent and conditioning (which is another way of saying it doesn’t have a Theory of Mind).

• And its ability to transfer knowledge is very crude compared to the natural way that human intelligence readily uses this skill (knowledge transfer is why you don’t have to retrain your young child to pay attention to trucks on the road after you’ve taught them to pay attention to cars). 

For Andreas Marksmann, a Danish journalist who took a yearlong fellowship to study the impact of automation in journalism, there are a lot of reasons to get excited about bringing in more automation and generative AI into our businesses. But we should be cautious in our adoption, even for simple articles or outputs. 

“When it comes to the end product, like service articles for instance, can we have a chat GPT powered-robot journalists write articles about football matches, for instance? I would recommend that people don’t do it at the moment with the technology we have currently,” Andreas told me in a recent chat, “because as folks who’ve studied this technology for a long time know, this technology sometimes has a problem distinguishing truth and lies. And it’s so hard for the technology to know the difference that sometimes it doesn’t even know when it’s lying.” 

AI’s lack of understanding of causality is seen as one of the most significant hurdles to clear by prominent researchers like Judea Pearl of Stanford. And knowledge transfer is a hurdle that stands between us and another new wave of improvement in deep learning, according to Andrew Ng of Baidu, formerly of Google Brain. (I’m linking to two important pieces that present their work, which, while not strictly related to news media, are just super interesting if you’re looking for a good, long read).

Artificial Intelligence, relative to the history of human progress and development, is actually still in its infancy. Think of the first steam engines of the early 18th century. Think about our modern engines that power spaceships that go into orbit.

Sure, technical advances do come faster than they used to because there is a compounded interest in our advancing skills and knowledge. And there is certainly plenty to be excited about. 

But the more you ask or read from these researchers who work at the bleeding edge of AI, the more you hear them acknowledge the reality that no matter how impressive our current stage of the journey, in reality, we are still at a very early stage in that journey.

The pitfalls are many, the aberrations frequent. Sure, a large-language model like GPT3 has read and digested a good chunk of the Internet and is able to converse pleasantly and competently with you about a number of things. But it will also fail at very basic tasks or behave at times entirely inexplicably.  

When we read reports of a conversion between ChatGPT and a NYT journalist (gift link) that seems to read like a Samuel Beckett play, we have to remember the very creators of these technologies absolutely agree that these are young systems. A top Google boss warned about their own chatbot’s hallucinations.

We overhype ourselves when we imagine we will soon see a world in which AI has replaced a large portion of a newsroom. If this is to even happen, this is a distant day.

Our first forays into generative AI should be as helper tools for humans 

Looking at our possible early applications for generative AI in news media, we need to remember our responsibilities in using these brand new, often rather green, tools.  

The work we do creating and publishing the news is, of course, of special import in society — the fourth estate, and all that. Even if the headlines about our new technical capacities seem to suggest that we’re on the cusp of a new revolution, the very specialists who have enabled the possibility of that revolution also readily tell us about all the ways in which these new tools and technologies are built on a partial understanding of the world, with significant holes to mind and gotchas to pay attention to.

“I think there are a number of different ways that we can use generative AI in journalistic work,” Andreas said. “One of them is as a tool, as a helper, you know, so it doesn’t produce the end product, but it helps us. For instance, I’ve used ChatGPT as an engine for generating and challenging headlines. So if we make a headline for a story, we can ask chatGPT to make 10 headlines that it thinks will work better.” 

While generative AI is only getting started, many news organisations have dabbled, with varying degree of intensity, into automated news over the years. This is for the most part far more mature, relying on rules and good structured data. 

“Basic automation can look like magic, but it’s very simple technology and doesn’t have any AI element. If you have structured data, then they can produce something today that can work quite well,” said Andreas, who also noted that, in general, when this type of automation went awry, it was more likely due to human error than issues with the technology. 

But with generative AI grabbing the headlines as it has, we have to be judicious in where we will build our first experiments. I was chatting with the lead for automation and data tools for a large North American media company recently who had been pulled into executive conversations that no longer questioned if generative AI was suited for experiments, but rather was focused on the priority to give to this work. 

The take of our automation lead was a variation on Andreas Markmann’s take — which was to give access to journalists to these new tools (rather than, strictly, to build new tools on top of them). Our American automation lead noted a rather strong enthusiasm based on folks approaching them to volunteer to be part of coming experiments — certainly something of note when so many gloomy headlines seem to suggest a cool, weary welcome from newsrooms.

Where Marksmann and their North American’s colleague approach have something in common is that both underscore the need to have a human controlling the output of the tool before it makes it to publication. And this approach also shares the same perspective that a good first round is informed by the feedback given by humans about these new technologies.

For our North American automation lead, “For now, [I want to approach these new tools] like: ‘What does this mean for us and how can we first understand it, react, and test how we can win in it and then scale it to the whole enterprise?’”

Further afield on the wide, wide Web 

• This one actually came out a couple of weeks ago but had to be cut for space in my previous newsletter. Still, it’s a good read, so, wouldn’t want to deprive you: The Economist looks at the new AI labs vying to come up with the latest AI technology (sorry, not a gift link).
• I linked in my post to the chat a NYT reporter had with ChatGPT in which ChatGPT seemed to get aggressive and unhappy as the conversation progressed. But there are other ways in which AI-driven systems can also affect our moods. In the MIT Technology review, a writer tells his story fending off algorithmic recommendations that kept driving them to sad articles as the writer researched stories about people losing their parents and how they managed to untrain the algorithms away from the content they wanted but also didn’t want anymore.
• The excellent data team at the German Bavarian state broadcaster, BR, took a gander on chatGPT taking the Abitur, which is the end-of-high school diploma examination. ChatGPT did well on certain things, but still couldn’t pass. Now, the Abitur is significantly harder than some of the basic questions you’d imagine Bing Chat could handle — and yet, when a Bing user asked it about the showtimes for the movie Avatar 2, it somehow led the AI to insist we were still in 2022.

About this newsletter

Today’s newsletter is written by Ariane Bernard, a Paris- and New York-based consultant who focuses on publishing utilities and data products, and is the CEO of a young incubated company, Helio.cloud. 

This newsletter is part of the INMA Smart Data Initiative. You can e-mail me at Ariane.Bernard@inma.org with thoughts, suggestions, and questions. Also, sign up to our Slack channel.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .