Description:

This workshop is influenced and partly derived from my PyDelhi workshop "Concurrency in the Python 3.0 world" given this year.

From my experience, most Python developers aren't still aware of the fundamental principles of concurrent programming, parallel computing and how to identify problems that yield well to data parallelilsm.

While the PyDelhi workshop focussed on introducing different types of concurrent programming techniques available in Python, this workshop wil be focussed more on concurrency using multiple processes - via the multiprocessing library and concurrent futures modules.

The focus is to enable the attendees to pick up the skills to write programs that scale to more than 1 CPU core.

We will start off with an introduction to concurency and parallelism plus why the GIL behaves the way it does with an example.

1. What is Concurrency - 5 min
2. Concurrency vs Parellelism - 5 min
3. The GIL weakness - Multithreading vs Multiprocessing - 10 min
   1. Prime number example with multiple threads & multiple processes
   2. Illustrating how GIL forces computing to 1 core

Next will be an introduction to Multiprocessing & Concurrent futures with some simple examples.

1. Quick introduction to multiprocessing - 10 min
   1. Concurrent futures - Introduction (10 min)
   2. ThreadPool vs ProcessPool executors

Now time for example problems which can be scaled using data parallel technique.

1. "Data Parallel" problems - 5 min
2. Matrix Multiplication - 20 min
   1. Serial
   2. Parallel version using Multiprocessing
3. Mandelbrot Fractals - 20 min
   
   1. Serial
   2. Parallel version using Multiprocessing
4. A simple web crawler - 20 min
   1. Serial
   2. Concurrent version using Multiprocessing
   3. Concurrent version using concurrent futures

[Break] - 5 min

Next I will demonstrate a Maze solver in Python and then show how to speed it up using parallel processing. (Note that this is experimental and maybe be replaced with something else in the actual talk !)

1. Maze solver in Python (30 min)
   1. Linear
   2. Speed up using Multiprocessing

Next the aspect of timing and measuring performance of your parallel/concurrent code.

(30 mins)

1. Why wall clock time is not all that matters
2. Using simple "time" command
3. Using "timeit" module
4. Debugging concurrent code

(If time allows)

1. Generators & concurrent futures - Some advanced examples - 15 min

Prerequisites:

1. Python programming fundamentals - Knowledge of generators is useful indeed.
2. Some awareness of Python GIL.
3. Some awareness of concurrent computing.
4. The code & discussions will be based on Python 3.x - the latest at that time.

This workshop is NOT for those who are just starting out as Python programmers.

Content URLs:

The latest workshop code and presentation is at.

https://github.com/pythonhacker/pyconindia2017concurrency

I may add more examples in the next few days but what is there now will not change!

Speaker Info:

Anand has been a long time advocate and community leader of the Python programming language in India and Bangalore.

Anand founded the BangPypers community in Feb 2005 as the result of a meeting of Bangalore Pythonistas. Discussions in this community went on to build other communities and laid the foundation of PyCon India and similar conferences.

He has 17+ years of software development experience having worked in a variety of technical roles in a number of software companies. He is currently working as Senior Architect at Yegii Inc., a startup from MIT, where he spends his time writing web crawlers to perform focussed and deep crawls for structured and unstructured data and to develop the next generation AI search engine for knowledge discovery.

His interests are high performance computing architectures, large-scale web crawling, information extraction & security.

Anand is also the author of a book discussing Python and Software Architecture for Packt Publishing, published in April 2017.

Speaker Links:

1. http://twitter.com/skeptichacker
2. https://github.com/pythonhacker