Real-Time Data Pipelines with Snowflake and Redpanda for Python Developers

Kamesh Sampath (~kameshsampath)




Streaming events can unlock a wealth of analytical insights through their payloads, but relying on streaming platforms as databases is a common pitfall. Data in isolation lacks value, requiring robust integration to be truly impactful.

Join us in this session as we explore how to seamlessly integrate Snowflake—the AI Data Cloud—and Redpanda, a super-fast streaming platform, to build a powerful data pipeline. We'll demonstrate how real-time application events can flow effortlessly into a data warehouse, becoming immediately available for analytics, machine learning, and data-driven applications.

You'll learn how to develop a scalable Python-based gRPC API to ingest data from Redpanda topics into the Snowflake AI Data Cloud. We’ll also cover how to visualize this data using Streamlit dashboards. By the end of this session, you’ll be equipped to build high-performance, real-time data pipelines that unlock the full potential of your data for advanced analytics and AI-driven insights.

Speaker Info:

Kamesh is a seasoned author and developer advocate with nearly two decades of experience in the tech industry. He is passionate about empowering developers by sharing cutting-edge knowledge, tips, and tricks. Specializing in data cloud technologies, Kamesh actively educates and inspires the developer community to achieve greater success. Throughout his career, he has played a key role in building robust enterprise solutions for a diverse range of clients in the services industry.

Speaker Links:

Section: Python in Web and Applications
Type: Talk
Target Audience: Beginner
Last Updated: