Celery for background task processing and deferred execution.
by konark modi (speaking)
Objective
Understand how we can use Celery in a web-app for background/asynchronous task which needs to processed immediately or after a scheduled interval.
Description
Celery is an open source asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well.
I this tutorial we will show the use of Celery and Django to build a simple scheduling web app that needs to process tasks immediately or after a scheduled interval.
The tutorial will be broadly based upon the following topics: 1. Celery architecture 2. Message broker (AMQP based messaging queue eg. RabbitMQ) 3. Connecting celery with RabbitMQ 4. Running Celery workers 5. Creating basic tasks and scheduling them. 6. Running Celery workers 7. Django Celery admin panel 8. Storing task results in a backend. (MySQL) 9. Monitoring the scheduling app for minimal downtime.
Setup Configuration: Intended Audience : *nix basics Basics of application development Basics of Python Fair assumption of Django architecture ( Though celery is not just for Django, but the demos would be based over Django)
System config. we'd be using: Centos 6.2 : Hosted on Virtual box Python version : Python 2.6.6 MySQL Django-1.4
Packages installed : 1. Virutal env : pip install virtualenv 2. RabbitMQ ( Recommended installation before Tutorial) incase internet is not working that particular day. 3. Celery : pip install -U celery 4. Django-Celery : pip install django-celery 5. MySQLdb module for python : easy_install MySQL , yum install mysql-devel python-devel 6. pytz : pip install pytz
Just installation would be sufficient as of now. We'd be understanding the components in detail during the tutorial.
Speaker bio
Piyush Kumar is Manager for Website Operations @ MakeMyTrip.com. He is an open source enthusiast who loves doing DevOps and Information security. His major work focus areas include : BigData, Monitoring, Handling Apache and Tomcat servers, MySQL, Django, F5 BigIP LTM/GTM, Profiling JavaApp using JProfiler/YourKit, Virtualization on Linux using XEN/VMware.
Konark Modi works as Sr. System Engineer for Website Operations @ MakeMyTrip.com. Major work focus areas include Designing/Writing/Implementing/Maintaining tools for monitoring/alerting purposes. Python/Django are the primary languages that is used in daily operations. Apart from this loves to interact with new technologies and keep updated with the latest happenings in the tech. space.
1
▼
Could you give the link for the sample code?