Description:

In the healthcare sector, accurate and comprehensive information retrieval is crucial for clinical decision-making, patient education, and research. Traditional text-based information systems often fall short in providing the necessary depth and context. The integration of multimodal Retrieval-Augmented Generation (RAG) systems offers a transformative approach by combining textual data with medical images, charts, and other relevant modalities.

This topic explores how multimodal RAG can enhance healthcare information systems. The discussion will cover the following aspects:

1. Clinical Decision Support: Multimodal Inputs: How clinicians can input both symptoms (text) and diagnostic images (e.g., X-rays, MRIs) to receive more accurate and contextually relevant diagnostic suggestions. Enhanced Retrieval: The process of retrieving relevant case studies, medical literature, and treatment guidelines that align with both textual and visual inputs. Generative Outputs: Generating comprehensive diagnostic reports and treatment plans that integrate findings from text and images.
2. Patient Education: Interactive Education Tools: Using multimodal RAG to create interactive tools that provide patients with detailed explanations of their conditions, combining textual descriptions with annotated images. Personalized Content: Tailoring educational material to individual patients by considering their medical history (text) and recent diagnostic images.
3. Medical Research: Data Fusion: Leveraging multimodal RAG to combine textual data from research papers with visual data from medical images to generate new insights and hypotheses. Information Synthesis: Generating comprehensive literature reviews and research summaries that integrate findings from both text and image-based studies.

Prerequisites:

Coding Background, Python,

Content URLs:

1. Linked id: https://www.linkedin.com/in/shubhamagnihotri17/
   2. Github: https://github.com/KillerStrike17
   3. Medium: https://medium.com/@shubham-agnihotri
   4. Portfolio Site: https://killerstrike17.github.io/

Speaker Info:

Shubham Agnihotri is a pioneering leader in the field of Generative AI, with over five years of extensive experience in Artificial Intelligence (AI) and Machine Learning (ML). Currently, he serves as the Senior Manager for Generative AI at IDFC First Bank in Mumbai, where he leads the development of cutting-edge Automatic Speech Recognition models for Indic languages and fine-tunes diffusion models for content generation. Shubham's expertise has been showcased at prestigious events, including Google DevFest, Google's flagship event; TechShow London, the UK's biggest tech event; and AWS Community Day, Amazon's flagship event.

At Google DevFest, Shubham captivated the audience by delving into the intricacies of transformers, the state-of-the-art ML model underpinning technologies like ChatGPT, teaching how to build these models from scratch. At TechShow London, he explored the profound impact of Generative AI across various industries, discussing its implications on different domains and the job market. During AWS Community Day, he highlighted the transformative effects of Generative AI on the finance industry, demonstrating his deep understanding of AI's real-world applications.

Shubham's impressive portfolio of projects includes developing a Retrieval Augmented Generative (RAG) model at Arcadis, which improved accuracy by 30% and enhanced user experience. He also spearheaded the creation of an AI-powered water utility tool, collaborating with cross-functional teams to save 25% in costs and increase efficiency by 20%. His innovative work on a Big Data ETL workflow enabled the processing of 4 billion data points in under 5 minutes for 11 clients. Additionally, Shubham automated workflows using Object Detection Models (YOLO) and Azure Cognitive Services, saving millions in costs and thousands of man-hours.

Previously, Shubham founded S.AgriUdaan, Gujarat's first agriculture drone service provider, where he developed a user-friendly marketplace platform connecting farmers with drone service providers, and delivered comprehensive agricultural services using UAVs. This initiative served over 15,000 acres and 4,500 farmers, and garnered partnerships with major clients like the Government of India, Adani, McCain, and others. His startup was also a finalist in Mahindra Startup Leap, a Mahindra and Mahindra initiative, where he had the opportunity to pitch his work to CEOs and CXOs of Mahindra Agri & Tractor Division.

In addition to his professional achievements, Shubham secured 2nd rank at the All India Police Hackathon by building a Facial Similarity and Recognition Algorithm for partially destroyed faces of corpses, developed for the Government of India, using Python and TensorFlow. His dedication to the AI community is evident through his volunteer work with the TensorFlow User Group Bangalore, where he hosted TensorFlow Everywhere India, TensorFlow's flagship event, and organized numerous events, reaching over 5,000 professionals. He also mentors students through Dreamers and Supporters, designing lectures and assignments in AI and Machine Learning.

Apart from these high-profile events, Shubham has spoken at various other events organized by TensorFlow User Group Bangalore and Mumpy - PyCon Mumbai chapter. He was specially invited to speak at Ramaiah University of Applied Sciences Bangalore, where he conducted workshops on AI, Python, and robotics, and at the Entrepreneurship Development Institute of India, encouraging students to pursue entrepreneurship. He has consistently been invited to speak at these events, showcasing his expertise and influence in the field. His commitment to education is further evidenced by his organization of workshops and sessions for both beginners and experienced professionals.

During his college years, Shubham developed a real-time facial recognition and tracking system using Raspberry Pi, akin to the "God's Eye" from the Fast and Furious series, capable of recognizing and tracking people. Additionally, he created a smart and sustainable aquaponics farming system powered by solar energy, leveraging IoT and AI, using Raspberry Pi and Arduino.

Shubham's accomplishments include winning the Data Science India Hackathon, the 10 Days of ML Challenge, and receiving multiple performance awards at Arcadis. He is a TensorFlow Certified Developer with a robust skill set in Python, MySQL, Pytorch, TensorFlow, Langchain, and LlamaIndex. His technical expertise extends to databases like MongoDB, SQL, DeepLake, and CromaDB, and tools such as Git, GitHub, AWS, Azure, Docker, Blender, Photoshop, Premiere Pro, Arduino, Jetson Nano, and Raspberry Pi. His commitment to fostering talent began during his time at Ramaiah University, where he founded Cynergy, the university's official coding group, organizing workshops, sessions, and seminars on coding, robotics, and AI. Shubham also earned a silver medal at Ramaiah University, further demonstrating his academic excellence.

Shubham's academic background from the Indian Institute of Management and Ramaiah University, coupled with his hands-on experience and leadership in over 40 events and workshops, positions him as a leading voice in the AI and data science community.

Speaker Links:

https://www.youtube.com/watch?v=XihAhZQZtV4