Taking a Multilingual Conversational Engine to Production: Theory to Reality

karthikavijayanexpts


7

Votes

Description:

We built a multilingual virtual assistant to cover a diverse customer-base in a country like India, where more than a 100 languages are spoken. The conversational engine is designed to engage with users in several languages, at least in the order of tens. Solving this problem was non-trivial. There were learnings from different challenges while deploying this engine to production. In this talk, we will discuss the development and deployment of a multilingual virtual assistant for Indian languages using a Python-based framework, while crafting the assistants' responses with dynamic data integration (a) for user's queries and (b) at assistant's own initiation.

We designed a language-agnostic natural language understanding (NLU) pipeline for the conversational engine, using which the virtual assistant 'understands' customer queries in multiple languages. The dialog flow for the conversational engine was designed by envisioning potential engagement scenarios in customer support. Custom responses from the engine are crafted in multiple forms, with integration of user-specific systems of records. This enables the virtual assistant to ‘respond’ to customer queries with an evolutionary conversational flow in a personalized manner. Our efforts in developing this multilingual virtual assistant have resulted in 97% NLU accuracy in benchmark measurements, and the deployed version attracted 93% of 'Happy' feedback rating from customers expressing their satisfaction.

Outline of the talk (tentative)

  • Introduction (2 mins)
  • How the conversational assistant understands queries in multiple languages (6 mins)
  • How the conversational assistant generates custom responses (4 mins)
  • Challenges in going to production : examples from real-life scenarios (5 mins)
  • Solutions for addressing these challenges and monitoring on prod (7 mins)
  • Q/A (5 mins)

Takeaways

  • Learn about a language-agnostic multilingual NLU process pipeline
  • Insights into data-conditions that aid multilingual conversational engines
  • Understand challenges in deploying such an engine in production
  • Addressing dynamic nature of dialogue flow and personalized response generation
  • Periodic monitoring and feedback

Prerequisites:

Basic understanding of machine learning

Video URL:

https://drive.google.com/file/d/1TsSV3Idx3rYUMnulfKgJr1m9Aq1UrsGe/view?usp=drive_link

Speaker Info:

Speaker bio -1 Dr. Karthika Vijayan is a Solution Consultant at Sahaj Software. She has been conducting research in the field of conversational AI with voice and text data for almost a decade. Her research has been published in several journals and presented at various international conferences. Her expertise includes creating customized solutions for real-world business problems by designing composite machine learning pipelines.

Speaker bio -2 Shruti Dhavalikar is a skilled Data Scientist with over 4 years of experience in the field, currently working as a Solution Consultant at Sahaj Software in Pune, India. With a deep passion for data science and a strong understanding of data analysis, she thrives on designing models that build valuable business insights from data. She has successfully delivered end-to-end product cycles under Agile methodology, showcasing her ability to handle diverse tech stacks and ensure scalable, clean, and robust design. Passionate about her work, she actively contributes to research projects and tries to stay at the forefront of advancements in the field. She has published research papers at international conferences.

Section: Data Science, AI & ML
Type: Talks
Target Audience: Intermediate
Last Updated: