SHARQ - An API queueing system built at Plivo!
by Sandeep Raju (speaking)
- Technical level
The audience will get to learn about an API queueing system we built at Plivo. This talk sheds light on the existing queues available and why we chose to build SHARQ instead of using the existing ones.
The talk will cover the following sections in detail.
Plivo often sees a spike in traffic patterns from some customers (for example when some customers decide to run a large Call or SMS campaign). These spikes tend to overload our systems because the automatic scaling cannot be fast enough for such work loads. Under such a scenario, some customers start affecting API requests from other customers, thus bringing down the overall service quality. We do not want to have our customers take the trouble to rate limit at their end. Thus, to ensure service quality, we need a system to queue API requests and process them at a controlled rate.
Available Solutions & Motivation for writing SHARQ
This section talks about how we evaluated Celery, and other queues. Why these did not fit into our requirements and why we chose to write it from ground up.
Architecture and Algorithm
This section gives an overview of the implementation of SHARQ, how to use it and a live demo. We’ll start by discussing the architecture we chose for SHARQ and the tools we chose to build it. Then, I’ll discuss the rate limiting algorithm and the redis data structures we have used. At the end, I’ll demo how SHARQ works.
Benchmarks, Production Metrics & Limitations
This section will cover some of the benchmarks of SHARQ and its analysis, some metrics from our deployment of SHARQ in production. We’ll also discuss when to use SHARQ and some ideas on how to make it scalable and highly available.
Discussion with the audience and answering any questions.
I work as a Product Engineer at Plivo. I am the primary author of SHARQ. Python has been my primary programming language from the past 4+ years. I am an ardent follower of Free & Open Source software and love to hack on it in my free time. I am one the founding members and admin of the PES Open Source Community. As a part of this community, I have conducted sessions on various open source softwares and have mentored some projects.
Here are my social profiles,
My personal website,