SHARQ - An API queueing system built at Plivo!
by Sandeep Raju (speaking)
Objective
The audience will get to learn about an API queueing system we built at Plivo. This talk sheds light on the existing queues available and why we chose to build SHARQ instead of using the existing ones.
Description
The talk will cover the following sections in detail.
The Problem
Plivo often sees a spike in traffic patterns from some customers (for example when some customers decide to run a large Call or SMS campaign). These spikes tend to overload our systems because the automatic scaling cannot be fast enough for such work loads. Under such a scenario, some customers start affecting API requests from other customers, thus bringing down the overall service quality. We do not want to have our customers take the trouble to rate limit at their end. Thus, to ensure service quality, we need a system to queue API requests and process them at a controlled rate.
Available Solutions & Motivation for writing SHARQ
This section talks about how we evaluated Celery, and other queues. Why these did not fit into our requirements and why we chose to write it from ground up.
Architecture and Algorithm
This section gives an overview of the implementation of SHARQ, how to use it and a live demo. We’ll start by discussing the architecture we chose for SHARQ and the tools we chose to build it. Then, I’ll discuss the rate limiting algorithm and the redis data structures we have used. At the end, I’ll demo how SHARQ works.
Benchmarks, Production Metrics & Limitations
This section will cover some of the benchmarks of SHARQ and its analysis, some metrics from our deployment of SHARQ in production. We’ll also discuss when to use SHARQ and some ideas on how to make it scalable and highly available.
Q&A
Discussion with the audience and answering any questions.
Speaker bio
I work as a Product Engineer at Plivo. I am the primary author of SHARQ. Python has been my primary programming language from the past 4+ years. I am an ardent follower of Free & Open Source software and love to hack on it in my free time. I am one the founding members and admin of the PES Open Source Community. As a part of this community, I have conducted sessions on various open source softwares and have mentored some projects.
Here are my social profiles,
My personal website,
Links
- SharQ Server: https://github.com/plivo/sharq-server
- SharQ (Core Library that powers SharQ Server): https://github.com/plivo/sharq
3
▼
Where can we see the code?
1
▼
Thanks for your interest. I'll share the link to the code in a few weeks.
3
▼
@sandeep Updates?
1
▼
@jaseem, I have replied about this to Aravind below..
1
▼
@Kushal: Here are the links to the source code.
SharQ Server: https://github.com/plivo/sharq-server
SharQ (Core Library that powers SharQ Server): https://github.com/plivo/sharq
Let me know what you think! :) I'll update the full docs as soon as possible.
1
▼
@kushal, update: I'll share the code here on monday :) finishing up with the docs!
1
▼
Is this open source? If it isnt, then how can people who attend your talk use it?
1
▼
@aravind: Here is the code.
SharQ Server: https://github.com/plivo/sharq-server
SharQ (Core Library that powers SharQ Server): https://github.com/plivo/sharq
1
▼
Thanks Sandeep. I'm reviewing the code and will give it a spin today if I get a chance. By when did you think you might be able to update the docs?
1
▼
@aravind, I just wrote some minimal docs for you to try it out: https://gist.github.com/sandeepraju/918b104aa9846c650d75 If you have any questions, send me an email (sandeep080[AT]gmail[DOT]com) or tweet to me @sandeeprajup :)
1
▼
Hi Sandeep,
I've given it a spin. Unless I've misunderstood the use case, it wasnt clear to me how this was different from an API frontend for queueing jobs to a system like RQ.
Also, did you have some internal monitoring tools built on top of this to check the queue status?
-A
1
▼
Hi Sandeep,
I've given it a spin. Unless I've misunderstood the use case, it wasnt clear to me how this was different from an API frontend for queueing jobs to a system like RQ.
Also, did you have some internal monitoring tools built on top of this to check the queue status?
-A
1
▼
Hi Aravind,
Thanks for trying out SharQ!
The key difference here is the ability to dynamically set rate limit to each queue. SharQ can be configured to dequeue / process jobs with a specific time intervals.
To give you an example, let us assume you would want to send out notifications to your customers. Assume that for customer X, the notifications are to be sent out at 1 message / minute (even though there are 100 messages pending for him, he would want to receive at only 1 message / minute). For customer Y, 10 message / minute, and so on for N customers. And the customer should have the capability to change the rate at which he gets the notifications dynamically.
All these are possible in SharQ unlike RQ. The entire flow control mechanism needed different structuring of the queue in Redis.
SharQ has a metrics api. I think I have not updated the docs. I'm also planning to push new updates.. I'll do it in a few days. :)
Let me know if you have any more questions.
1
▼
Hi Aravind,
This indeed will be open sourced. I am working on making this open source and the plan is to get this out by the end of this month. It has been delayed as there were some things pending (like docs, test cases, etc.). I'll be sharing the links to the source code here in a few weeks.
1
▼
@aravind, update: I'll share the code here on monday :) finishing up with the docs!