Feel the Flow: A Journey into Distributed Processing
Jerry Thomas (~jerrythomas) |
Description:
"Feel the Flow: A Journey into Distributed Processing," is a focused introduction to the world of distributed processing, using Prefect Flow as the central tool. This session is designed to give attendees an insightful overview of distributed processing, while also detailing some functionalities and advantages of Prefect Flow. The aim is to provide participants with a solid introduction to these concepts, coupled with practical exposure to reinforce this knowledge.
This workshop will guide participants through the core components of Prefect Flow, including tasks, states, and flows, and demonstrate how to construct a basic Prefect Flow. The workshop will also feature a case study to provide a real-world context for these applications. With this hands-on, step-by-step approach, participants will gain theoretical knowledge and practical skills in distributed processing using Prefect Flow. By the end of the workshop, attendees should feel more confident in exploring and utilizing this powerful tool in their own projects.
"Feel the Flow: A Journey into Distributed Processing," is a focused introduction to the world of distributed processing, using Prefect Flow as the central tool. This session is designed to give attendees an insightful overview of distributed processing, while also detailing some functionalities and advantages of Prefect Flow. The aim is to provide participants with a solid introduction to these concepts, coupled with practical exposure to reinforce this knowledge.
This workshop will guide participants through the core components of Prefect Flow, including tasks, states, and flows, and demonstrate how to construct a basic Prefect Flow. The workshop will also feature a case study to provide a real-world context for these applications. With this hands-on, step-by-step approach, participants will gain theoretical knowledge and practical skills in distributed processing using Prefect Flow. By the end of the workshop, attendees should feel more confident in exploring and utilizing this powerful tool in their own projects.
Workshop Outline:
- Setting Sail: Introduction (15 minutes)
- Welcome and introduction to the journey
- Brief discussion on the importance of distributed processing and its evolution
- Discovering the Vessel: Understanding Prefect Flow (30 minutes)
- Introduction to Prefect Flow and its role in distributed processing
- Highlighting the unique features and benefits of Prefect Flow
- Brief interactive Q&A to clarify concept
- Reading the Compass: Basic Concepts of Prefect Flow (45 minutes)
- Explanation of the tasks, states, and flows
- Understanding how Prefect Flow organizes and executes workflows
- Interactive exercises and Q&A for participants
- Sailing the Seas: Hands-on Tutorial - Building a Prefect Flow (60 minutes)
- Guided demonstration of creating and orchestrating tasks
- Observing and managing flow state
- Hands-on activity for participants to construct a basic Prefect Flow for distributed processing
- Brief Q&A and troubleshooting session
- The Treasure Trove: Distributed Processing Real-World Case Study (15 minutes)
- Reviewing a real-world case study showcasing the effective application of Prefect Flow in distributed processing
- Analyzing the approach, implementation, and outcomes
- Docking at Port: Extended Question & Answer Session (30 minutes)
- Opportunity for participants to ask in-depth questions, seek clarifications and discuss potential use cases
The workshop will adopt an interactive, journey-oriented approach, enabling participants to not only understand Prefect Flow and distributed processing but also experience it first-hand. By the end of this journey, participants will be well-equipped to chart their own courses in the vast sea of distributed processing.
Prerequisites:
Prerequisites for the Workshop:
- Basic Programming Knowledge: Participants should have a basic understanding of programming concepts and some hands-on experience with a language, preferably Python, since Prefect Flow is implemented in Python.
- Understanding of Data Processing: Some familiarity with data processing workflows would be beneficial. This includes understanding how data can be ingested, processed, and outputted in a computational context.
- Familiarity with Cloud Concepts: As the workshop involves discussions around distributed processing, some basic knowledge about cloud concepts and services can be advantageous. This includes understanding the idea of distributed systems, cloud computing models (like IaaS, PaaS, SaaS), and basic cloud operations.
- Working Environment: Participants should have a working Python environment set up on their computer to follow along with the hands-on demonstrations. A code editor (like Visual Studio Code or PyCharm) and a recent version of Python installed would be necessary.
- Optional - Knowledge of Workflow Management Tools: While not strictly necessary, having some experience with or knowledge of other workflow management tools (like Apache Airflow or Luigi) can provide additional context and understanding.
Please note, this workshop is designed to be an introduction to Prefect Flow and distributed processing, so even if participants are not completely familiar with all the prerequisites, they should still be able to gain valuable knowledge and experience from the session.
Speaker Info:
Jerry Thomas is an experienced architect and mentor who has worked in various technologies and domains for over two decades.
His focus lies in providing architectural and technological guidance across various domains, assisting teams in understanding and executing open-source projects effectively. With a wealth of international experience, Jerry has collaborated with clients from different geographical locations, fostering a broad understanding of global technological landscapes. His curiosity and adaptability make him a quick learner, always ready to master and incorporate new technologies into his work.