Maxing out on I/O performance with Asynchronous Python
Ashwin Udayakumar (~ashwin84) |
Description:
Multi threading is one of the most common approaches in optimising an I/O bound task. However, multi-threading in Python has its own limitations due to its dependency on the operating system for the context switch. Since the programmer has no control over the context switch, multi-threading doesn’t always result in max performance. Asynchronous Python(AsyncIO) enables programmer to take such control when writing concurrent code. Even though this requires the libraries to support asynchronous version of their functions, asyncIO opens doors to achieve massive level of I/O concurrency owing to its independence from the Operating System for context switch. The talk introduces python internals and concepts involved in asynchronous programming and ways to optimize the performance by employing multiprocessing along with asyncIO.
Outline:
- Asynchronous programming - what it is? (8)
- Preemptive vs cooperative multitasking
- Pausing and resuming functions in python
- How asynchronous scheduling works (4)
- Synchronous vs asynchronous version (5)
- Limitations
- Optimisations? (8)
- Using multiprocessing along with asyncio
- Optimising further with some preprocessing, map/reduce
- Benchmarks
Who is this talk for:
- Python programmers who deal with concurrency
- Anyone who is interested to learn about concurrency concepts and asynchronous programming
Takeaways:
- Understand how asynchronous computing works in python and in general- benefits and trade-offs
- Understand how multiprocessing and asyncIO can be combined for maximum performance
Prerequisites:
Core python internals
- Some knowledge on Operating system and concurrency
Content URLs:
Speaker Info:
Ashwin solves engineering problems at MadStreetDen, Chennai.