+1 -1 +51
Vote on this proposal

SHARQ - An API queueing system built at Plivo!

by Sandeep Raju (speaking)

Section
Infrastructure
Technical level
Intermediate

Objective

The audience will get to learn about an API queueing system we built at Plivo. This talk sheds light on the existing queues available and why we chose to build SHARQ instead of using the existing ones.

Description

The talk will cover the following sections in detail.

The Problem

Plivo often sees a spike in traffic patterns from some customers (for example when some customers decide to run a large Call or SMS campaign). These spikes tend to overload our systems because the automatic scaling cannot be fast enough for such work loads. Under such a scenario, some customers start affecting API requests from other customers, thus bringing down the overall service quality. We do not want to have our customers take the trouble to rate limit at their end. Thus, to ensure service quality, we need a system to queue API requests and process them at a controlled rate.

Available Solutions & Motivation for writing SHARQ

This section talks about how we evaluated Celery, and other queues. Why these did not fit into our requirements and why we chose to write it from ground up.

Architecture and Algorithm

This section gives an overview of the implementation of SHARQ, how to use it and a live demo. We’ll start by discussing the architecture we chose for SHARQ and the tools we chose to build it. Then, I’ll discuss the rate limiting algorithm and the redis data structures we have used. At the end, I’ll demo how SHARQ works.

Benchmarks, Production Metrics & Limitations

This section will cover some of the benchmarks of SHARQ and its analysis, some metrics from our deployment of SHARQ in production. We’ll also discuss when to use SHARQ and some ideas on how to make it scalable and highly available.

Q&A

Discussion with the audience and answering any questions.

Speaker bio

I work as a Product Engineer at Plivo. I am the primary author of SHARQ. Python has been my primary programming language from the past 4+ years. I am an ardent follower of Free & Open Source software and love to hack on it in my free time. I am one the founding members and admin of the PES Open Source Community. As a part of this community, I have conducted sessions on various open source softwares and have mentored some projects.

Here are my social profiles,

My personal website,

Comments


  • 3

    [-] Kushal Das 268 days ago

    Where can we see the code?


  • 1

    [-] Aravind Krishnaswamy 227 days ago

    Is this open source? If it isnt, then how can people who attend your talk use it?


    • 1

      [-] Sandeep Raju 209 days ago

      @aravind: Here is the code.

      SharQ Server: https://github.com/plivo/sharq-server
      SharQ (Core Library that powers SharQ Server): https://github.com/plivo/sharq


      • 1

        [-] Aravind Krishnaswamy 207 days ago

        Thanks Sandeep. I'm reviewing the code and will give it a spin today if I get a chance. By when did you think you might be able to update the docs?


        • 1

          [-] Sandeep Raju 206 days ago

          @aravind, I just wrote some minimal docs for you to try it out: https://gist.github.com/sandeepraju/918b104aa9846c650d75 If you have any questions, send me an email (sandeep080[AT]gmail[DOT]com) or tweet to me @sandeeprajup :)


          • 1

            [-] Aravind Krishnaswamy 166 days ago

            Hi Sandeep,

            I've given it a spin. Unless I've misunderstood the use case, it wasnt clear to me how this was different from an API frontend for queueing jobs to a system like RQ.

            Also, did you have some internal monitoring tools built on top of this to check the queue status?
            -A


          • 1

            [-] Aravind Krishnaswamy 166 days ago

            Hi Sandeep,

            I've given it a spin. Unless I've misunderstood the use case, it wasnt clear to me how this was different from an API frontend for queueing jobs to a system like RQ.

            Also, did you have some internal monitoring tools built on top of this to check the queue status?
            -A


            • 1

              [-] Sandeep Raju 166 days ago

              Hi Aravind,

              Thanks for trying out SharQ!

              The key difference here is the ability to dynamically set rate limit to each queue. SharQ can be configured to dequeue / process jobs with a specific time intervals.

              To give you an example, let us assume you would want to send out notifications to your customers. Assume that for customer X, the notifications are to be sent out at 1 message / minute (even though there are 100 messages pending for him, he would want to receive at only 1 message / minute). For customer Y, 10 message / minute, and so on for N customers. And the customer should have the capability to change the rate at which he gets the notifications dynamically.

              All these are possible in SharQ unlike RQ. The entire flow control mechanism needed different structuring of the queue in Redis.

              SharQ has a metrics api. I think I have not updated the docs. I'm also planning to push new updates.. I'll do it in a few days. :)

              Let me know if you have any more questions.


    • 1

      [-] Sandeep Raju 227 days ago

      Hi Aravind,

      This indeed will be open sourced. I am working on making this open source and the plan is to get this out by the end of this month. It has been delayed as there were some things pending (like docs, test cases, etc.). I'll be sharing the links to the source code here in a few weeks.


    • 1

      [-] Sandeep Raju 213 days ago

      @aravind, update: I'll share the code here on monday :) finishing up with the docs!

Login with Twitter or Google to leave a comment →