Large scale python deployments using native packages.
by dj (speaking)
Objective
Rebuilding virtualenvs in each server is pain. Using natives packages
is much better and scalable.
Description
Here is the brief outline of talk
Problem
git clone && pip install becomes painful when you scale.
- Its slow.
- You are wasting precious CPU cycles to do things again and again
- Each server should have the build toolchain.
- Wont' play nice with config managers.
We are trying to solve something thats been solved by apt,
yum and essentially all sane package management systems.
Inspiration
This talk and workflow is heavily inspired from a blog post by Hynek Schlawack.
However the original post used needed too much boilerplate code.
Solution
[Disclaimer: Talk becomes specific to debian from here]
Well, we can just use the package manager. But not all programmers are familiar with debian
packaging. But all python programmers (almost) understand setup.py. Using a wrapper (dh_virtualenv) that connects best of both can solve this.
Speaker bio
I work as an infrastructure engineer with plivo. At work I manage large scale deployments mostly in python. I have been using python for last six years.