Chatbots and Text Classification

Practical advice on intents

The problem of recognizing intents in dialogues is, after all, a text classification problem. This article on Medium, apart from reintroducing the concept, gives some practical advice on how to deal with this problem.

Assuming we choose the machine-learning approach, and our goal is first of all to identify intents, these are things we should take into account

• Intent count — average number of intents for one app should be 5–10 intents. Fewer intents will be too simplistic, while more intents will harm the accuracy.
• Data Magnitude and quality— we all know that as in any machine learning task, the more data we have, and the closer it is to inference queries, we will have better results
• Transfer learning possibility — transfer learning, or in other words, using a pretrained model for similar problem may be very helpful if available
• Inference input size — users don’t tend to be concise in their querying of our bot. therefore, text summation tools, among with cues for users to be short and to the point, will help our app to have better accuracy

Few intents, lot of data, force users to write short sentences. And reuse a pretrained model.Got it ?

The Classifier - the core of the Chatbot

In the end, the problem of finding intents for queries is a text classification problem. You might use standard sci-kit pipelines, combined with text preprocessing, to create such classifiers. The approaches used to take on problems such as the google newsgroups dataset still work.

However, it is possible that we are not going to have anywhere near 20000 texts to train our classifier. We might then use at first the "Wechat" approach : start from keywords and then move on to machine learning, leveraging on the data that we gather from users.

What Pipeline to use

Ok, sure, this is all nice and good. However, text classifications problems are not all born equal. First and foremost, you have to think on how to clean up your text.

This article suggests a natural pipeline for chatbots. This sums up the preprocessing part:

• Spellcheck 
• Split into sentences 
• Split into words 
• POS tagging 
• Lemmatize words  
• Entity recognition 
• Find concepts / Synonyms  

Once you have normalized your sentences you can move on to next step ?

Rule Based or Machine-Learning ? 

The above mentioned article goes on to suggest defining patterns or DSLs to identify intents from the simplified sentence. That is a Rule-Based approach, pretty much the Pattern matching approach we already mentioned.

Machine-learning approaches may give better results when more data is available.

Chatbots : Intents or Entities ?

A general architecture of chatbots can be found in this article. Required reading, if you want to understand chatbots. As of now, the most popular open-source framework to create chatbots is RASA, which is made of RASA_NLU and RASA_CORE.

The NLU (National Language Understanding) component

A chatbot must have a NLU component, whose purpose is to identify the intents and the entities of user queries.

Intents : the bots must be able to understand what the intention of the user is. Whether he wants to make a reservation at a restaurant, for instance.

Entities: In the context of Bots, these are neither "Nouns" nor Named Entities that may be recognized by a NLP library such as Spacy (though these may overlap). An entity is an actionable piece of information.

For instance, if the user says : I want to reserve a table for two at Stacy's, tonight at 19:30 , the intent would be make_reservation, while the entities would be two, Stacy's and 19:30. We can use this information to book a table for two at Stacy at 19:30. However, for Christ's sake, table in this context is not an entity, it is actually part of the intent that we identified, make_reservation (at a restaurant)!

Intents or Entities?

 While it would be great to have a system which would be able to recognize a good amount of intents and entities, in practice you see that successful chatbots choose to concentrate either on one or the other.

Prioritize Intents ? : the typical use case of this is when you want your bot to answer users' Frequently Asked Questions. If the expected action is just give a canned answer once you recognized an intent, you don't need entities at all. 

Prioritize Entities ? : this is a typical scenario for the most focused (and usually most successful) bots. Usually these are the bots who drive the conversation so that they know exactly what intent to expect at a certain juncture. For instance:

ChatBot: Hi,  I am the Stacy's restaurant chatbot. Choose your action:

1. Make a Reservation

2. Cancel a Reservation

(User picks 1)

Please tell me about your reservation.

User: I want to reserve a table for two, tonight at 19:30

In this case, the chatbot knows that the user's next sentence will be about reserving a table, so it will just concentrate on getting the entities. Even easier is decomposing the query even further.

ChatBot: Hi,  I am the Stacy's restaurant chatbot. Choose your action:

1. Make a Reservation

2. Cancel a Reservation

(User picks 1)

For how many people?

User: Two
When?

User: Tonight, at 19:30

In this case, forget about intents, drive the conversation and just train your bot to recognize entities.  

Bootstrapping Chatbots

It is ok to bootstrap your chatbot using keywords / hand-coded rules, but you should think of ways to gather data from users interaction and leverage it using machine learning.  

 

Introducing WeChat

I found this article about the experience learnt while creating the WeChat chatbots, in China, of all places.

WeChat started creating simple chatbots in 2009. WeChat proves that, as a matter of fact, chatbots can be successful. China lives in another internet ecosystem and may show us an alternative timeline, compared to the one we are living in the west. In our "Western" timeline, chatbots are nowhere as successful as they are in China.

Too bad I don't speak Chinese and cannot actually try these chatbots.

How is WeChat doing ?

WeChat has made it easy to create very simple chatbots that are actually not very intelligent. It has become one of the favored ways for customer service to reduce the work they do interacting with customers online.

 Chumen Wenwen ... has built a very sophisticated bot that ... combines voice recognition, AI and the WeChat platform into a package that queries information for its users. By connecting with third party API’s the app can answer questions about what is around you, including movies, restaurants, massage and more.

We are astonished to learn that many entrepreneurs doing business in China create an official WeChat account before they even launch their website.

Customer service accounts

The kind of bots we are interested in are customer service accounts. 

They turned out to be necessary in China because customers

bombard the brand themselves with questions, before they make a purchase online or in-store. The sheer number of consumers and their zest for questioning is what created the need for chatbots in China before they ever came into the minds of Facebook and others.

 

Nevertheless, it is important to point out that bots are incapable of handling all types of customer requests. As a result these accounts need to be supported by employees.
 

Technology

Let's get to the part that interests us most - technology. 

WeChat bots work by identifying keywords in text strings and using hand-coded rules for how to respond to different situations. Yet, they still use machine learning and the more users interact with companies and brands via chatbots, the smarter they will become, and the faster they will learn to understand user inputs.

Lesson To Learn: it is ok to bootstrap your chatbot using keywords / hand-coded rules, but you should think of ways to gather data from users interaction  that you can leverage using machine learning.

What do these bots have to do anyway ? They have mainly to answer users' queries, not to take actions on them. So my guess would be that they are focused on intents rather than on entities.

Although bots are gradually learning to understand customer requests, they still text back only pre-set responses.

So after they have recognized users' intents, they text back precanned-responses. Got it.