Communications - Natural Language Understanding

In Chapter 5 we look at communications, and in particular at Natural Language Understanding as it is so key to a virtual human. On this page we'll introduce the main approaches to natural language understanding, and link to places that you can try different methods out, and papers to help you build your own, or understand what is going on. Note that most of these systems also have some form on in-built natural language generation system built around a template model, our page on Natural Language Generation will look at this area in more detail, and in particular at non-template approaches.


Mimic Bots


This is probably the simplest form. The user says something to the chatbot, it doesn't know how to reply so says something generic. Then when talking to another user (or ideally users) later it asks them the same question, stores their answer, and then has its answer for next time someone asks! Apart from being very pattern specific (although ML might be able to fix that) it has the big issue that if used unsupervised the bot will have a very disjointed set of knowledge and may take on some very strange or even racist, sexist and hate filled views. Just ask Microsoft about Tay.

Simple Pattern Matching


Here author creates a set "cases" - each essentially a pattern-response pair. The pattern is defined in some form of markup so it can include wildcards and synonyms. The system then finds the best/only/closest/first matching pattern and uses the appropriate response.

The most well known pattern based chatbot system is Richard Wallace's Artificial Intelligence Markup Language (AIML) from and you can log on to Pandorabots to make your own AIML bot. Compilers for AIML have been created for a wide variety of computer languages. See:


Through commands like the <system> tag an AIML bot can reach out to web services to help answer queries, but there is limited capability for more complex scripting.

Also check out:


Rule-based Pattern Matching


The next step up from AIML is systems like Chatscript. They are "rule based" as a case/rule will only fire if a number of conditions are met (unlikely the normal tree approach of AIML - although that can be extended) and also rules can be set to mute once used to avoid bot repetition. The best known rules-based system is Bruce Wilcox's Chatscript which has a scripting language that can be used in the body of each case, so quite (very) complex rules and actions can be made. Chatscript also includes an internal triples base for knowledge storage. Check out:

Also:


Machine Learning Based Systems


Machine learning systems take a set of example inputs and then work out the salient features that link them to a particular intent and/our desired output. This seems to be where most of the current research work is - but you may find that the practicality of using them is a long way from the hype. Unlike the above two approaches were you use some markup to define the pattern variations you are after with an ML system you just type "loads" of examples. In fact they only ask for 5-20 examples per "intent" - which seems to limit the ML scope. The authoring tools also seem either very primitive (as list of intents) or very restrictive (drag-and-drop style). Context needs to be manually managed (like AIML) in most cases, and they are typically very much based on "intent" - what specific things does the user want to do (e.g. book a flight) and so what information does the bot need - they are not really designed for open conversation. It's dead easy to try them for yourself as most allow for free trial use. Check out:


Useful papers:
  • Jaffe, E., White, M., Schuler, W., Fosler-Lussier, E., Rosenfeld, A., & Danforth, D. (2015). Interpreting Questions with a Log-Linear Ranking Model in a Virtual Patient Dialogue System. Silver Sponsor, 86-96. The Twelfth Workshop on Innovative Use of NLP foBuilding Educational Applications Stroudsburg, PA: The Association for Computational Linguistics
  • Khanna, A., Pandey, B., Vashishta, K., Kalia, K., Pradeepkumar, B., & Das, T. (2015). A Study of Today’s AI through Chatbots and Rediscovery of Machine Intelligence. International Journal of u-and e-Service, Science and Technology, 8(7), 277-284.
  • Vinyals, O., & Le, Q. (2015). A neural conversational model. Proceedings of the International Conference on Machine Learning, Deep Learning Workshop. Available online https://arxiv.org/abs/1506.05869

Grammatical Parsers


As far as we can see the only real way forward with natural language for virtual humans is to get a proper grammatical parser, which understands sentences word by word (or phrase or idiom) as humans do. Whilst there used to be a lot of research in this area it seems that a lot of researchers switched to AIML (if they just wanted a chatbot for an experiment) or Machine Learning, so finding decent up-to-date work in this area is hard. We've dabbled a bit, and will continue, but one good example bot is:
  • Arckon - http://artistdetective.com/arckon/
It uses slightly more triples based approach than pure part-of-speech, but is still a good step forward.

Useful papers include:
  • Good question!


The Loebner Prize and Turing Test


You can't talk much about chatbots and natural language without talking about the Turing Test and the Loebner Prize annual implementation of it. They are still good tests of chatbots, even if not of intelligence, but can suffer from the issue that everyone (expect the bot) knows that they are taking part - hence our work on covert Turing Tests. Relevant papers and links are:

  • Bradeško, L., & Mladenić, D. (2012). A survey of chatbot systems through a loebner prize competition. In Proceedings of Slovenian Language Technologies Society Eighth Conference of Language Technologies (pp. 34-37).
  • Burden, D. J. H., Savin-Baden, M., & Bhakta, R. (2016). Covert Implementations of the Turing Test: A More Level Playing Field?. In International Conference on Innovative Techniques and Applications of Artificial Intelligence (pp. 195-207). Cham: Springer.
  • Gilbert, R. L., & Forney, A. (2015). Can avatars pass the Turing test? Intelligent agent perception in a 3D virtual environment. International Journal of Human-Computer Studies, 73, 30-36.
  • Savin-Baden, M., Tombs. G.,  Burden. D., & Wood, C. "‘It’s Almost Like Talking to a Person’: Student Disclosure to Pedagogical Agents in Sensitive Settings." International Journal of Mobile and Blended Learning, 5 (2), (2013): 78-93.
  • Warwick, K., & Shah, H. (2016). Can machines think? A report on Turing test experiments at the Royal Society. Journal of experimental & Theoretical artificial Intelligence, 28(6), 989-1007.

Problem Chatbots


As well as Tay there are various other situations where chatbots can get a bad press:

  • Cockayne, D., Leszczynski, A., & Zook, M. (2017). # HotForBots: Sex, the non-human and digitally mediated spaces of intimate encounter. Environment and Planning D: Society and Space, 35(6), 1115-1133.
  • Newitz A (2015, 27 Aug) The Fembots of Ashley Madison. Gizmodo. Available online: http://gizmodo.com/the-fembots-of-ashley-madison-1726670394

Conversation Design and Analysis


Probably the biggest issue with almost all of the above systems is that they are focused on input-response pairs, rather than trying to create a bigger context of managing a whole dialogue (beyond simple intent based slot filling). Even just detecting and matching speech-acts seems beyond most systems. So alongside the move to a grammatical based system there also needs to be far more attention paid to conversation and dialogue design, creating the more two-way and asymmetric conversations of real life. For instance one of our own bots is quite happy to rabbit on for ages if you just no your head each time - and the more he talks the more he drifts from the original subject - just like a human! There is also some useful work around "argumentation" which is all part of conversation design.

Useful readings:


  • Bibauw, S., François, T., & Desmet, P. (2015). Conversational agents for language learning: state of the art and avenues for research on task-based agents. CALICO edition. Boulder, CO, USA. Available online  https://lirias.kuleuven.be/bitstream/123456789/499442/1/2015.05.28+Dialogue+systems+for+language+learning.pdf.
  • Nolan, B. (2014). Extending a lexicalist functional grammar through speech acts, constructions and conversational software agents. In B. Nolan & C. Periñán-Pascual (Eds.),
  • Sidnell, J. (2011). Conversation analysis: An introduction. Hoboken, New Jersey:  John Wiley & Sons.
  • Thimm, M., Villata, S., Cerutti, F., Oren, N., Strass, H., & Vallati, M. (2016). Summary report of the first international competition on computational models of argumentation. AI magazine, 37(1), 102.
  • Wei, B., & Prakken, H. (2017). Defining the structure of arguments with AI models of argumentation. College Publications 68  1-22  Available online https://dspace.library.uu.nl/bitstream/handle/1874/356057/WeibinPrakken17.pdf?sequence=1







Subscribe to: Posts (Atom)