Thinking in Functions
Anoop Thomas Mathew (~atmb4u) |
30
Description:
Introduction
Use some of the Functional Programming concepts applicable to Python, to write concurrent and scalable Python code.
Synopsis
Python is NOT meant to be a functional programming language[1]. At the same time, writing functional programs has been the most efficient way to build scalable systems (Eg: WatsApp, RabbitMQ). This puts the developer in a hard position to choose from. There has been enough systems in the ecosystem to enable users to make user of functional programming in Python. But, there are very limited resources for programmers to select what matters them most and make use of functional concepts within python language.
Many of the functional programming aspects like tail recursion and currying are not directly available in python, and the workshop doesn't intend to make python a complete functional programming language [2]. At the same time, making use of many functional concepts while writing Python code will enable it to be run parallel and some concurrently in turn making it more scalable.
Attendees will be working on a live project to learn how to think each functionality as functions, and separate data objects to make sure of the separation of concerns and how to minimize side effects. Workshop will discuss on modularizing programs and principles of separation of concerns. It will also touch up on map-reduce and minimizing side effects.
Topics under Discussion
Basics of Functional Programming
- Function as the atomic entity
- Side Effects
- Immutability
- Lazy Evaluation
Functional methodology in Python
- itertools and functools
- Decorators and Generators
- What python CAN do - Eg: lazy eval, lambda
- What python CANNOT do - Eg: Tail Recursion, Pure immutability, Pure Functions
Thinking about scalability with functional methods
- map-reduce
- shared states
- Immutable 'variables'
- independent functions
- Pipelining
- Chaining
- Functional Patterns
Performance Comparison of Functional Programs vs Object Oriented Programs
- The curse of GIL - workarounds
- Scaling to n number of machines
- %timeit and other benchmarks
- CPU intensive processes vs I/O intensive processes
Hands On: Building a data analytics pipeline - A functional approach
- Creating stages in using applying functional concepts in Data Gathering, Data Cleansing, Data Analysis and Post Processing
- Creating auto scalable solution similar to Apache Storm [Update: Due to time constraints, we might not do this along with the workshop]
- Using Functions only to create pipelines
Intended Audience
Ideal attendees would be the ones who are deeply interested in designing and building scalable applications, but is still confused over the concepts of functional programming, and is also enthusiastic about how to use functional programming aspects in python.
Goals and Key Takeaways
Goal of the workshop is to make sure that attendees are be able to create programs that are a mix of imperative along with functional paradigm and learn which all parts can be run concurrently, which others can be run parallel and how to do that in most Pythonic way possible.
Prerequisites:
Basic knowledge in Programming, Python and UNIX.
Content URLs:
Slides for the proposed workshop : Thinking in Functions (on progress)
Slides for a few sessions the speaker has presented:
Faster Python @ PyCon
Ambidextrous Python @ DevCon
Prototyping Painkiller Startups
Getting Started with git @ FOSSMeet, NITC
Speaker Info:
Anoop Thomas Mathew has been in the industry for past 5 years, and been involved in designing and developing large scale concurrent systems to solve real world problems. He is a Python Architect with years of experience in working with python and software development. With the title of Chief Technology Officer @ Profoundis, he leads the engineering efforts at Profoundis and the technology architect of vibe. Vibe handles 100,000+ users and more than 100 million requests every month. Concurrency and parallelism has been at the core of the Vibe architecture. He has spoken at numerous conferences like Fifth Elephant 2012, PyCon 2012, PyCon 2013, FOSSMeet 2014, DevCon. He's an avid contributor to open source (link) and contributed to django and cPython.