+1 -1 +31
Vote on this proposal

Gevent and Coroutines - Scaling Real World Network Applications

by Aalok Sood (speaking)

Section
Web Development
Session type
Talk
Technical level
Intermediate

Objective

Illustrate how gevent and coroutines make writing highly concurrent real-world python applications a breeze.

The aim of the session is to make network/server application authors appreciate the scalability of gevent and coroutines as an alternative to multithreaded and multiprocess models.

Description

Many real world applications are I/O bound. Network can quickly become the bottleneck. The traditional approach to developing concurrent networking servers is to write a multiprocess or a multithreaded server.

The multiprocess model (especially in python) is quickly limited by available memory (& sometimes cpu) as multiple copies of a fat interpreter are running. The multithreaded model in Python suffers from the GIL. Also, the continuous context switching in threaded processes is a huge overhead. Deadlocks & race-conditions come with the territory. As we all know, threads are not the solution, threads are the problem.

In this talk I will introduce coroutines (greenlets) along with the powerful gevent library (greenlets for network processes). In my experience gevent based servers scale the best. I'll talk about a real world application used by very large enterprises and its evolution from being a multiprocess to multithreaded to a gevent core. I will describe the limitations we hit with the first two models and detail the subsequent rewrite of the server using gevent. I will also describe the ease with which the code was ported to the gevent model.

Speaker bio

Born in the peaceful valley of Manali, I am a true nature's child. My life revolves around logic, motorcycles, sports and counter strike.

I am a Senior Member of Technical Staff at Idea Device (http://www.ideadevice.com) Bangalore. To this date, I hold the record for the maximum bugs introduced in our software.

Comments


  • 1

    [-] Rajendra Prasad.k 876 days ago

    Hi.. Talk was good. Can you please share materials. Thanks in advance.


  • 1

    [-] Aalok Sood 875 days ago

    Thank you everyone for the great response. I have uploaded the slides too :)

Login with Twitter or Google to leave a comment →