Event-Driven Application using Python & Apache Kafka
hemi s.k (~hemi) |
Description:
The workshop provides participants with insights into building event-driven applications with Apache Kafka and Python. By the workshop's conclusion, attendees will have acquired knowledge of Kafka's problem-solving capabilities, learned to implement Kafka in their projects, integrated it with Python libraries, and utilized Kafka Connect for seamless event routing across systems. Covered topics include distributed systems, Kafka event handling, read-write operations, partitioning, scaling, and consumer group management. This workshop is designed for engineers and architects seeking to enhance their applications with real-time data integration functionalities. Teaching methodologies encompass code demonstrations, practical examples, and actionable guidance for incorporating Kafka effectively. Workshop Syllabus:
Introduction
- Overview of the Workshop
Event-Driven Applications
- Understanding Event-Driven Architecture
- Importance and Benefits of Event-Driven Design
Introduction to Kafka
- What is Apache Kafka?
- Key Concepts of Kafka
- Understanding Kafka's Distributed Nature
Kafka Events
- Exploring the Structure of Kafka Events
- Writing Events to Kafka
- Reading Events from Kafka
Pizza Demo!
- Interactive Demonstration: Ordering Pizza with Kafka
Managing Kafka Logs
- Log Size and Retention Policies
- Understanding Kafka Topic Partitions
- Selecting an Appropriate Partition Strategy
- Ensuring Data Ordering within Partitions
Scaling Kafka
- Strategies for Scaling Out Kafka
- Partition Management and Scaling Considerations
Partitions Demo!
- Hands-On Exercise: Working with Kafka Partitions
Coordinating Multiple Applications
- Introduction to Kafka Consumer Groups
- Consumer Group Strategies and Best Practices
Consumer Groups Demo!
- Practical Session: Implementing Consumer Groups in Kafka
Kafka Connect
- Overview of Kafka Connect for Data Integration
- Use Cases and Configuration Options
Additional Resources
- Recommended Reading Materials, Tools, and Online Resources for Further Learning
Prerequisites:
- Basic programming concepts
- Familiarity with Python programming
- Understanding of distributed systems concepts
- Interest in event-driven architecture
- Desire to learn Kafka
Speaker Info:
Hello, I'm Hemangi Karchalkar. Armed with six years of professional programming experience and an enduring love for coding, I presently serve as a Senior Software Developer at EPAM Systems. Beyond my coding prowess, I find fulfillment in sharing knowledge with others. My passion lies in aiding individuals in their journey to success in the realms of data science and artificial intelligence. I've been fortunate to present at esteemed gatherings such as PyCon Italia 2023, Pycon India 2023, Pyconf Hyderabad 2022, Pycon Italia 2024, and GIDS Bangalore 2023.