Building highly scalable web-services with Gevent - experiences at Plivo
by Aditya Manthramurthy (speaking)
Objective
This talk will introduce Gevent, how it works and how it can make Python scale. Plivo's architecture consists of multiple web-services built on Flask and Django. The talk will outline scaling challenges faced at Plivo, and how we handled them using Gevent with Flask and Django to achieve high throughput and high CPU utilisation.
From this talk, attendees can expect to have an understanding of Gevent that will help them build highly concurrent webservers, especially with Flask or Django, often with little changes to existing code and without having to learn a complicated callbacks-based abstraction as used in frameworks like Tornado.
Description
Gevent gives your Flask/Django application the ability to do asynchronous I/O with little or no modifications! This lets your Python web-server scale to over thousands of threads without running out of memory - C10K with Python, we're nearly there!
Audience
Web developers using Django or Flask or a similar framework, who would like to understand how Gevent can help them get much higher concurrency, often with little or no changes to existing code.
Talk Outline
-
An outline of scaling and concurrency challenges at Plivo, and how Gevent helped us solve these problems (~5 minutes)
Blocking I/O, multi-process scaling limitations, spiky workloads and low CPU utilisation were major bottlenecks, that we were able to mitigate and solve using Gevented web-servers.
-
Gevent concepts: Co-routines, event loops, greenlets and monkey-patching (~5 minutes)
Gevent is a co-routine based networking library that uses light-weight execution units, called greenlets, to provide a high-level synchronous API on top of a fast event loop provided by libev.
Monkey-patching provides a way to convert existing Python Standard Libraries to be greenlet aware, so they yield execution when they start to do IO.
Gevent switches threads of execution, either when the running greenlet yields explicitly, or when the greenlet performs I/O. This requires that libraries doing I/O in an application, are either pure Python or provide a way to be "greened" (for e.g. the PostgreSQL database driver psycopg2).
-
How can I tell if Gevent will help my application to scale (~3 minutes)
Things to factor in while making the decision: Is the application IO bound or CPU bound? Are "green" alternatives available for libraries doing IO?
-
Demo to show Gevent performance and ease of use with Flask (~5 minutes)
After an outline of request processing in a typical web-application, we simulate a real-world web-server deployment using Flask and see how performance improves when using Gevent.
-
Porting applications to use Gevent: what to lookout for (~6 minutes)
Learnings from porting a large Django application to Gevent - on debugging blocking operations, database drivers, and libraries that cannot be "greened" easily.
-
Questions (~6 minutes)
Speaker bio
I am a Software Developer at Plivo. I've been programming in Python for over 6 years. At Plivo, I work on scaling and infrastructure management. We use Gevent extensively in our production systems, and I have gained a lot of valuable experience using it, especially with porting a large Django code-base to use gevent.
1
▼
This sounds interesting. I've seen issues with mixing Flask with async stuff (gevent-socketio), and I'd be curious to see to what extent you've used gevent beyond basic greening. Could you please provide links to slides and videos of talks that youve given in the past? This would help people decide whether they want to attend.