Unlocking the Power of Ragas: A Framework for evaluating Retrieval Augmented Generation (RAG) Pipelines

Sumit Jaiswal (~justjais)




Join me as I introduce Ragas, a revolutionary framework designed to help you evaluate and optimize your Retrieval Augmented Generation (RAG) pipelines. In this presentation, I'll delve into the world of RAG, exploring its applications in Language Learning Models (LLMs) and the challenges associated with pipeline performance evaluation.

Presentation Highlights:

  • An introduction to RAG and its role in LLMs
  • Insights on the importance of evaluating RAG pipelines and common challenges faced by developers
  • A deep dive into Ragas, a framework designed to assess and optimize RAG pipeline performance
  • Real-world examples and case studies demonstrating the effectiveness of Ragas


  • A comprehensive understanding of RAG and its applications in LLMs
  • Insights on the importance of evaluating RAG pipelines and common challenges faced by developers
  • Knowledge on how to use Ragas to optimize your RAG pipeline performance
  • Inspiration for applying Ragas to your own projects and improving their overall efficiency

Who Should Attend:

  • NLP researchers and developers interested in RAG and LLMs
  • Industry professionals looking to improve the performance of their language-based applications
  • Anyone seeking to stay up-to-date with the latest advancements in natural language processing


  • Though not a hard requirement/pre-requisite, but it is recommended that attendees have a basic understanding of Natural Language Processing (NLP), Machine Learning, and Information Retrieval concepts. As, RAG is an advanced topic that combines these three areas to generate human-like responses to text or speech queries by using machine learning models, knowledge graphs, and large datasets.
  • Having a understanding of the mentioned technologies will enable you to fully engage and follow along with the discussions during the talk. However, if you are new to these concepts, attending the RAG talk can also be an excellent opportunity for you to expand your knowledge and gain insights into this rapidly evolving field. Also, there's no requirement of having an understanding of finance, as LLM should be able to explain everything in simple human readable format.
  • Lastly an open mind and a desire to learn new technologies focused on evaluating and improving AI performance. :-)

Speaker Info:

  • With over a decade of experience weaving my way through the intricate web of technological advancements, I, Sumit Jaiswal, have journeyed from the depths of device drivers to the thrilling heights of AI and machine learning.
  • I am a open-source enthusiast and enjoy experimenting with different open-source technologies. Currently residing in Noida, I've previously lived in Bangalore for more than a decade and fondly miss the city's weather and its Biryani.
  • I am a adventurous soul, who loves traveling and engaging in discussions about emerging technologies throughout the day.

Speaker Links:

  • As, I am part of RedHat Ansible organization I actively contribute towards writing blog around Ansible and its applications, ref: https://www.ansible.com/authors/sumit-jaiswal/
  • I've delivered multiple talks at several conferences like AnsibleFest, DevConf, and Config Management Camp. All of the talks are available at their respective youtube channels.
  • LinkedIn profile: https://www.linkedin.com/in/sumit-jaiswal-justjais-6a480467/

Section: Artificial Intelligence and Machine Learning
Type: Talk
Target Audience: Intermediate
Last Updated: