Parallelism in Python

Description:

Python as a language has enjoyed a decade of usage in both industry and academia. It has been the most popular abstractions to scientific computing and machine learning, yet the base remains single-threaded. Just how is productivity in these fields being maintained with a single-threaded language?

Python has a terrible reputation when it comes to its parallel processing capabilities. Ignoring the standard arguments about its threads and the GIL (which are mostly valid), the real problem with parallelism in Python isn't a technical one, but a pedagogical one. The common tutorials surrounding Threading and Multiprocessing in Python, while generally excellent, are pretty “heavy.” They start in the intense stuff and stop before they get to the really good, day-to-day useful parts.

The talk will highlight the impact of Global Interpreter Lock on multithreaded programs and include code snippets to compare single and multithreaded programs using a real-life example. Then, we move on to the concept of multiprocessing and the address its benefits over multithreading and mention some examples for it. The talk will end with an application of multiprocessing in the area of Computer Vision and how it speeds up the completion of a task by 82%.

Basic Outline of the talk

• Introduction and Overview of Python's parallel processing capabilities [2-3 minutes]
• Is multi-threading a scam in Python? [2-3 minutes]
• Global Interpreter Lock [5 minutes]
• Impact of GIL on multi-threaded programs
  • Code Example and Walkthrough
• Introduction to Multi-processing [4 minutes]
• I/O Bound, CPU Bound Tasks and the tradeoff [3 minutes]
• Real life example of Multi-processing [3 minutes]
• Q/A [5 minutes]

Who is this talk for

• Python developers who deal with performance issues on a daily basis.
• Hobbyists who're looking to improve the performances of their python projects.
• Anyone who's curious about GIL and its controversies.

Key Takeaways

• Understanding how the GIL works and affects Python Programs.
• A fresh perspective on when to use multi-threading and when to use multi-processing.
• A deep dive into the inner workings of python.

Prerequisites:

• Basic Scripting in Python.
  • Knowledge of multi-threading and multi-processing.

Content URLs:

The talk is inspired by Real Python's Blog Post and this talk by Mike Muller on Shared Memory in Python.
Talk Slides and Code will be available on GitHub .

Speaker Info:

My name is Rounak Vyas, I’m currently a junior at SRM University Chennai, India majoring in Information Technology. I’m a student researcher at Next Tech Lab and also a member of the Artificial Intelligence Research Team in the Association of Computing Machinery - SRM Chapter. My area of research involves working with computer vision models, multi-agent reinforcement learning, I’ve been a speaker at two tech conferences in India: PyData KTR and ChennaiPy, talking about the parallel processing capabilities of python.

Speaker Links:

GitHub - https://github.com/itsron717
LinkedIn - https://linkedin.com/in/itsron143