The curious case of slow/fast grequests code

Saurabh Hirani (~saurabh)


3

Votes

Description:

If you have made a single HTTP request in Python, you have probably used the requests module.

If you have made concurrent HTTP requests in Python, you have probably used the grequests module.

I ran into an issue in production which made a grequests based code execute HTTPS GET requests serially without utilising the concurrency goodness promised by grequests. What was more interesting is that depending on the Python modules installed on my system, the same codebase ran fast or slow.

Tumbling down that rabbit hole made me understand Python profiling, function tracing, understanding how gevent works and opening a pull request to fix the issue with a gevent based Python module.

In this talk, I want to share my learnings with the audience through a working dockerized demo which showcases the following stages for the same codebase while explaining why each stage behaves the way it does:

  • Stage-0 - Trigger which led to the exploration - May run slow/fast for Python 2.7 and slow/fast for Python 3.7 depending on the modules installed on your system.
  • Stage-1 - Baseline with dockerized environment - Runs fast both for Python 2.7 and Python 3.7
  • Stage-2 - Detection of slowness and verification - Runs slow both for Python 2.7 and Python 3.7
  • Stage-3 - Python 2.7 fix - Runs fast for Python 2.7 but slow for Python 3.7
  • Stage-4 - Python 3.7 fix - Patching a gevent based Python module which fixes the issue and makes Python 3.7 fast again.

Please check out the preview video to get a sneak peek into the issue and watch the same codebase run slow/fast depending on the Python version chosen.

Prerequisites:

Python basics.

Speaker Info:

  • Saurabh Hirani normally doesn't talk about himself in third person, but when he does, he is writing a speaker bio.
  • He enjoys programming and infrastructure automation.
  • He is currently working as a Site Reliability Engineer at Autodesk and has spoken at Python meetups, Rootconf conferences and firmly believes in sharing hard earned knowledge that everyone can benefit from.

Section: Core Python
Type: Talks
Target Audience: Intermediate
Last Updated:

Hello Saurabh,

Thanks for submitting the proposal, it looks quite interesting. Please add the slides too when they're ready. Here are some more best-practices you might want to check out - https://github.com/pythonindia/junction/wiki/Speaker-best-practices

Regards,
Abhishek
(CFP Co-ordinator)

Abhishek Yadav (~zerothabhishek)

Thanks for the best practices Abhishek. I will check them out and add a preview video. I will work on the slides and upload them by next week. I had a question around them - as slides will be work in progress with basic structure being constant and delta changes happening over time, is that fine? Or is there is a deadline after which slide editing is not permitted?

Saurabh Hirani (~saurabh)

Thanks for the best practices Abhishek. I will check them out and add a preview video. I will work on the slides and upload them by next week. I had a question around them - as slides will be work in progress with basic structure being constant and delta changes happening over time, is that fine? Or is there is a deadline after which slide editing is not permitted?

Saurabh Hirani (~saurabh)

Hi Saurabh, The idea is to have sufficient content before the expert reviewers start reviewing. Having the slides before that would be good. We're planning to start with the expert reviews in about a two weeks (May first week).

Abhishek Yadav (~zerothabhishek)

Thanks Abhishek. Added preview video. Slides are work in progress. Will add them and update this thread.

Saurabh Hirani (~saurabh)

Awesome !

Abhishek Yadav (~zerothabhishek)

Hi Abhishek - added first cut slides.

Saurabh Hirani (~saurabh)

👍🏼

Abhishek Yadav (~zerothabhishek)

Login to add a new comment.