Writing a BitTorrent engine - powered by asyncIO
arpit. oberoi (~arpit.) |
Description:
NOTE: I say BitTorrent ENGINE and not a client because it is not your average libtorrent client, it's a BEP compliant asynchronous BitTorrent engine written from scratch, in python, powered by asyncIO.
AsyncIO is a python library released as a part of standard library in python 3.5.
It is powered by an event loop that supports uvloop, which is a cython wrapper over libuv, The same library that powers node.js.
3 releases later, asyncIO has still not been accepted by the larger half of the community. The problem being the absence of obvious use cases and therefore the sheer lack of interest.
The asyncIO library came out boasting of IO performance comparable to node.js and goroutines but in 'make 10^N web requests' benchmarks which doesn't intrigue everybody except maybe someone who wants to take a website down.
Finding new asynchronous alternates for all your favorite libraries(requests) isn't fun either.
The microservices, RESTful, basically any interactor based architectural patterns won't make it any easier.
In this talk i will demonstrate how making some design choices and understanding the real reason why your traditional approach to some problems might not work out when you start out with asyncIO.
Basic outline of the talk
- How single threaded IO achieves the performance it boasts [4-5 minutes]
- Common roadblocks on the way to asynchronous awesomeness [5 minutes]
- Why asynchronous, Favorable vs Forced use cases [4-5 minutes]
- How projects like aiohttp and sanic do what they do to achieve what they achieve [2 minutes]
- BitTorrent, the ideal candidate for this talk [5-10] minutes
- Event driven architecture [5 minutes]
- Q/A [2-5 minutes]
Who is this talk for ?
- People who don't want to create a separate node.js/golang microservice every time they see IOPS heavy use cases.
Key takeaways
- Solution design
- Event driven architecture
- Writing beautiful asynchronous abstractions
Details
Addressing the elephant in the room, Why BitTorrent ?
Simple answer: Because it looked complicated and challenging myself to do this would result in me spending some quality time with asyncIO
Real answer: Some of the main operations performed by a Bit Torrent engine are downloading data, writing to disk, reading from disk and uploading data. All these operations are Network IO or Disk IO operations which makes it a perfect candidate for asyncIO.
Bit Torrent was something i was always interested in, reading wiki articles to understand how it works and does what it does. Everything about Bit torrent, from binary encoded metadata files to Peer wire protocol and finally to some extensions that are worth mentioning here, like mainline DHT, everything has always attracted me towards it.
Implementing a protocol from scratch was on the bucket list for quite a long time now and Bit Torrent being a perfect AsyncIO candidate just made it happen.
Note: This talk extends ahmed abdalla's talk and fixes/completes his Bit Torrent implementation using asyncIO
Prerequisites:
- Python
- Networking basics
- P2P basics
- BitTorrent basics
Content URLs:
https://docs.google.com/presentation/d/1EH2XGwCm4C8SPhlbz26lBCjlM31nOoMJUcC0-NDG8dA/edit?usp=sharing
http://github.com/supersaeyan/bittorpy
Speaker Info:
Arpit Oberoi has been a python developer for more than 4 years now and has been professionally designing scalable solutions for a more than a year.
Areas of interest:
Linux enthusiast
ProudParselTongue(python)
Performance oriented python(cython, asyncio)
Always interested in clean code practices, scalable architecture designs, design patterns, style guides
Speaker Links:
EMAIL: arpitkoberoi@gmail.com | arpit.oberoi@cloudsek.com
Blogs
- Blog post 1 https://medium.com/@arpitkoberoi/cyberoam-bot-871441a2418f
- Blog post 2 https://medium.com/@arpitkoberoi/fortinet-bot-d90fdb08b25e
- Blog post 3 https://medium.com/@arpitkoberoi/latest-tv-show-episodes-and-where-to-find-them-59288224528b
SOCIALS
- https://github.com/supersaeyan/
- https://medium.com/@arpitkoberoi
- https://mobile.twitter.com/Arpit_Saeyan
- https://www.facebook.com/supersaeyan
- https://www.instagram.com/really_bad_influence/