Gevent and Coroutines - Scaling Real World Network Applications
by Aalok Sood (speaking)
Objective
Illustrate how gevent and coroutines make writing highly concurrent real-world python applications a breeze.
The aim of the session is to make network/server application authors appreciate the scalability of gevent and coroutines as an alternative to multithreaded and multiprocess models.
Description
Many real world applications are I/O bound. Network can quickly become the bottleneck. The traditional approach to developing concurrent networking servers is to write a multiprocess or a multithreaded server.
The multiprocess model (especially in python) is quickly limited by available memory (& sometimes cpu) as multiple copies of a fat interpreter are running. The multithreaded model in Python suffers from the GIL. Also, the continuous context switching in threaded processes is a huge overhead. Deadlocks & race-conditions come with the territory. As we all know, threads are not the solution, threads are the problem.
In this talk I will introduce coroutines (greenlets) along with the powerful gevent library (greenlets for network processes). In my experience gevent based servers scale the best. I'll talk about a real world application used by very large enterprises and its evolution from being a multiprocess to multithreaded to a gevent core. I will describe the limitations we hit with the first two models and detail the subsequent rewrite of the server using gevent. I will also describe the ease with which the code was ported to the gevent model.
Speaker bio
Born in the peaceful valley of Manali, I am a true nature's child. My life revolves around logic, motorcycles, sports and counter strike.
I am a Senior Member of Technical Staff at Idea Device (http://www.ideadevice.com) Bangalore. To this date, I hold the record for the maximum bugs introduced in our software.
1
▼
Hi.. Talk was good. Can you please share materials. Thanks in advance.
1
▼
Thank you everyone for the great response. I have uploaded the slides too :)