Towards a Method to Classify Language Style for Enhancing Conversational Systems

Abstract

Chatbots have received significant attention in the last years. These systems have improved operational efficiency by reducing the cost of customer service and more and more are used in customer service channels. More recently, with the addition of speech capabilities added to those systems, bot language modifications have to be done in order to improve user experience. In this paper, we analyze the language used in those systems using datasets collected from real chatbots. For that, we first propose models that are able to identify the language style (writing, speech, and computer-mediated) using as linguist features syntactic (Biber's dimensions) and sentence embeddings as well as state-of-art datasets. Our results show that our models were able to distinguish among the three classes with an accuracy of up to 66%. Finally, we evaluate real chatbots systems, either speech- and text-based ones, using our proposed models. We found that these real chatbots are generally using a computer-mediated style, but there is indication that their developers tend to adapt the language style to be more speech-like when text-to-speech is used.