Modern python parallelism using actor models as a threading alternative
Praveen M (~praveen13) |
Description:
Threading is one of the most well-known approaches to attaining Python concurrency and parallelism. Decades of untraceable bugs and developers’ depression have shown that threads are not the way to go. It has several problems like: deadlocks, bad scaling, shared state. Many solutions were suggested to solve such problems. Just like locks and mutexes. But all of these are related to synchronization. Shared state makes lots of problems, it involves the idea of mutating the memory. And this is fine as long as you only have one process doing the mutating. But if you have multiple processes sharing and modifying same data — this is gonna be a recipe for disaster. There are some tricks of synchronization and locking mechanisms.
Use of actors allows us to: Enforce encapsulation without resorting to locks. Use the model of cooperative entities reacting to signals, changing state, and sending signals to each other to drive the whole application forward. Stop worrying about an executing mechanism which is a mismatch to our world view.
Outline of the talk:
- Multithreading in python:
- How and why
- Problems with GIL
- What is an Actor Model
- Implementation & Benifits
- Fault Tolerance through Encapsulation
- Actor as a Supervisor
- Achieving Parallelism with Actor Model
- How to achieve meaningful concurrency
- Code Example and walkthrough
- Implementation on a Distributed System
- Multithreading vs Actor Model
- Illustration
- Benchmarks
- Orchestration over a Cluster of Compute Nodes
Breakdown:
- Multithreading and GIL - 5mins
- Introduction to actor model - 5mins
- How the actor model meets the needs of modern, distributed Systems -10mins
- Multithreading vs actor model - 5mins
- QnA - 5mins
Who is this talk for:
- Avid Python Programmers who deal with parallelism
- Just about any python programmer who's keen on interpreting real world learnings
Key Takeaways:
- How GIL works, what an interpreter level lock actually does
- Limitations and Performance barriers with threading<->GIL
- How functional programming efficiently attains parallelism
- How to attain meaningful parallelism and concurrency in python
Prerequisites:
- Understanding of multithreading and multiprocessing
- Object oriented programming
Content URLs:
Modern python parallelism using actor models as a threading alternative: Link
Speaker Info:
Praveen has been a pythonista since 2014, preaching python over multiple forums such as FSFTN, GLUG and meetups!.
Currently scaling AI to millions at MadStreetDen.