In March of this year, Microsoft CEO Satya Nadella boldly announced, “Bots are the new apps.” In the months since, dozens of news publishers — The Washington Post included — have tested that thesis, launching bots on platforms like Facebook Messenger, SMS, and Alexa. With the year coming to a close, it’s a good time to reflect on these experiments.

What have we learned? Where are we going?

One thing that became evident early in our work with bots is that many people who interact with them don’t even know it.

“What’s a bot?” That was the question my friend, Ben, asked when I invited him to try out The Washington Post’s new “Feels” bot.

The question took me by surprise. The news and tech industries have been talking about bots all year. Surely, everyone knew what a bot was, right?

Nope, Ben didn't know what bots were. Turns out, neither did my mom, neither did my brother, and — as I saw in the conversation logs of our experiments — neither did many of our own bot users.

“Bots” is not a marketing-friendly word. Users don’t know what bots are. Perhaps, after a few bot-ty smash hits, the term will be common parlance, but it’s not right now.

In fact, I probably shouldn’t assume you know what a bot is either. 

By definition, bots are:

Conversational programmes …

You use words, either spoken or typed, to interact with them. Ideally, you converse in as natural language as possible.

... that can be distinct entities with personalities ...

You need a something or a someone to converse with; some experiences are anthropomorphised with names, avatars, even unique senses of humor.

... providing information or doing tasks on your behalf ... 

Schedule a timer. Check a sports score. Get the latest headlines. Sound familiar? You’ve probably already done this using bots like Siri.

… often living on third-party platforms.

If you don’t own a hardware platform, you’re hosting your bots in someone else’s ecosystem like Twitter, Facebook Messenger, or Alexa.

Launching our own bots

At The Washington Post, we launched several bots during the past year.

• Washington Post on Facebook Messenger: Our first bot was a simple one. Users could message “Washington Post” on Facebook to see top headlines, ask for stories about a topic, and sign up for election night alerts. During Olympic season, we added the ability to ask about schedules, event results, and medal tallies.

• Debates SMS: We’ve been wanting to experiment with SMS for a long time. It might be the most ubiquitous platform outside of the Web. We recently finalised a new SMS short code and tested it out with a presidential debates experience. Readers could opt into live debate fact-checking and analysis from reporter Aaron Blake. His real-time commentary was peppered with charts, animated gifs, and emoji — just like you were chatting with a buddy of yours.

• “Feels” Bot: Our most recent bot was our most ambitious. For three weeks before the U.S. presidential election, our Feels bot checked in with users every day with one simple question: “How is the election making you feel today?” The bot delivered a morning graph of user feelings and excerpts of why users felt that way. While hosted on our Washington Post Politics page, Feels was wrapped up in a welcoming, friendly persona.

Through the launch of these three bots, we learned many lessons that led to some difficult conversations.

Part of the excitement driving bots is the increasing maturity of machine learning, artificial intelligence, and natural language processing (NLP).

Unfortunately, there’s a large disconnect between those concepts and the day-to-day realities of bot platforms.

Many NLP libraries are vaporware or just over-hyped. They don’t integrate elegantly into bot platforms and they don’t do much work on your behalf. Set-up often involves labouriously cataloging all of the ways a user might ask a specific question and manually associating it with a specific feature. It’s a lot more artificial than intelligent.

We saw how this can go wrong in our Washington Post Facebook Messenger app. The open-ended interface invited users to ask questions beyond the scope of our feature set, resulting in frustrating, “Sorry, I can’t help you” responses from the bot.

We also learned that finding the right Olympic events via natural language (“did you mean male/female?” “individual or relay?” “50m/100m/200m?”) is more difficult than just presenting users with an old-fashioned clickable list of events.

We applied some of those lessons learned on Feels. First, we narrowed the scope of the app to checking in on emotions and reporting back what users said. We reinforced that feature set with specific, consistent prompts every day. Guided conversational paths reduced exploration and produced more satisfying check-ins.

The Power of the push

We saw some clear usage trends in our first Facebook Messenger app. Users came, tried it out, and never came back. Understandably so.

The app didn’t offer features different from our other apps or Web site. And the experience — typing back conversational sentences — was less productive than more traditional interactions like lists, buttons, and Google searches.

But bots on messaging platforms have one key strength and differentiator: the ability to automatically reach out when they have something to say.

Push notifications are powerful. In our post-debates survey, 68% of participants said they read every single text during the debate. In Feels, we averaged more than one-third of participants answering the daily prompt every single day.

In multiple conversations with users about using these platforms, they said they felt an urgency seeing an unread number next to their messaging app icons. Those channels are usually where their friends and family reach them.

Using push notifications allowed us to tap into existing communication habits, those informing how they keep in touch with those closest to them all day long. (Though who knows how that will last, if and when these channels get more populated with publisher bots.)

Onboarding, scale, and social

There are multiple hurdles getting users to sign up for a bot. Again, most users don’t know what bots are. They aren’t used to automate experiences in channels that are mostly for personal connections.

Marketing and onboarding processes should reinforce the value proposition and depict the user experience clearly. We can’t rely on users having a mental model of these experiences, like they do for mobile apps.

We also have a lot to learn about how to grow our audience in these channels.

In other channels, it’s relatively easy for users to be ambassadors for our content. A Washington Post e-mail newsletter can be forwarded. An article on washingtonpost.com can be shared on Facebook and Twitter.

It’s not clear right now the best way to make bot content and interactions social.

Individual messages seem too “atomic” to share. They need more context. Do we need to build in corresponding Web experiences for bots with dynamic URLs that are share-able and contextual? I anticipate this will be a point of evolution for bot platforms like Facebook Messenger in the future.

Are bots the new apps? No, not yet.

The industry didn’t see any breakout bot hits in 2016. Traditional Web and native apps have too many advantages over bots. I suspect we’re still years away from commodity technology that unlocks the real power of conversational interfaces.

Meanwhile, we’re going to continue running better and better experiments. We hope you’ll join us along the way.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .