Making 1k OpenAI calls to GPT-3.5/4 models in under 5 minutes with asyncio - the right way!
Santhosh Kumar Dhanasekaran (~santhosh_kumar) |
Description:
Using LLMs in the enterprise has become widespread and have started realizing the true power of LLMs it has become effortless with Azure OpenAI Service that guarantees user data are NOT available to other customers, NOT available to OpenAI, are NOT used to improve OpenAI models and much more, these are important for using enterprise data with LLMs.
The important piece to solve for enterprise-grade is speed, as the models get larger and stronger, they tend to take more time to process the user prompt and generate a completion for it. We’ll also go over a few simple ways to make the completion generation faster.
After this hands-on talk you'll be able to:
- Generate embeddings at lightning speeds (hours to just minutes)
- Great understanding of the power of asyncio
- Use your tokens wisely and avoid some completion to reduce cost
- Write better Python code for interacting with Azure OpenAI Services
Content:
- $whoami [5 minutes]
- Quick introduction: asyncio and LLMs [5 minutes]
- KISS with asyncio [3 minutes]
- Live Demo on making 1k OpenAI API Calls [5 minutes]
- Optimizations for completion [4 minutes]
- Support from Azure OpenAI Services [3 minutes]
- Q&A [5 minutes]
Prerequisites:
- Basic knowledge of Python’s asyncio library
- Understanding of large language models (LLMs) and their usage
Speaker Info:
Santhosh Kumar aka Sandy is a Data Engineer at Rakuten, has an everlasting curiosity to try out new technologies, and shares his experience at community events.
Delivered four conference talks
- Microsoft London office (Power Automate Summit 2023)
- Microsoft Bangalore office (Global Azure 2023)
- Microsoft Chennai office (Microsoft 365 Saturday 2022)
- The Leela Palace, Chennai, India (Az Community Conference – Asia’s Largest Azure Community Conference – 1000+ in-person attendees)
10+ sessions for Azure Developer Community
Won ten hackathons participating Solo (~$14,600) including four from Microsoft Azure.
Microsoft Certified Trainer (MCT) since 2021, teaching about Azure AI capabilities, and has the following certifications under his belt DP-203, DP-100, PL-900, AI-900, DP-900, AZ-900
- Lead for Azure Developer Community - Chennai region.
Once a coder, Always a coder - Sandy