Description:

In use cases involving manipulation of large amounts of data, like scientific computing applications, copying data across libraries or different objects, for example from a numpy array to a pytorch array can be expensive. In these cases, it will be better to share a pointer to the underlying memory instead of explicitly copying the data. To this end, Python needs a mechanism to share a view of the underlying data across the libraries and at the same time, capture the complexities of the data, like data type, shape and dimensions. Here comes the Python buffer protocol. The talk delves into the need for buffer protocol, the mechanism provided by PEP 3118 for sharing buffers and discusses how it is implemented in practice.

Outline of the talk:

• Buffers and Arrays (3 min)
• Motivation for the buffer protocol (3 min).
• Discussion on PEP-3118 (3 min)
• Technical details about buffer protocol implementation (7 min).
• Code demo on using the buffer protocol (10 min). (total: ~25 min)

Talk takeaways

The key takeaway for the audience is the introduction of Python C-API to beginner audiences in a simple easy-to-follow demo and for intermediate/advanced users, it introduces the buffer protocol. The Python C-API provides rich and powerful functionalities which can be leveraged in applications like scientific computation and to access the python interpreter. Apart from it, the other takeaways are:

• Understand the difference between a buffer and an array.
• Get to know the technical details of the buffer protocol.
• Get hands-on experience with the buffer protocol via a code-demo.

Prerequisites:

Familiarity with C programming language.

Video URL:

https://drive.google.com/file/d/1uakT_UHwd9_ICqXvaLLrFWCkGfN0bJiX/view?usp=sharing

Content URLs:

• Related blog on the post written by Arun: https://www.arunppsg.in/posts/buffer-protocol-consumer/
• Draft slides: click here

Speaker Info:

Arun is a machine learning software engineer working on building web-based machine learning platform for computational drug discovery at Deep Forest Sciences. He is a contributor to the open-source library DeepChem, a scientific python library for applying deep learning in molecular data. Before Deep Forest, he worked as a project assistant at RISE Lab, Indian Institute of Technology, Madras for about a year where his work focused on detecting malwares from network traffic using predictive algorithms. Arun is also an active volunteer at FOSSUnited, a non-profit aimed at strengthening the open-source software ecosystem in India by helping the team organise the monthly FOSSUnited meetups at Bangalore. His technical writing can be found in his webpage and his code at github.

Speaker Links:

• Lightning talk in PyCon India 2022 available here
• Github: https://github.com/arunppsg/
• Webpage: https://www.arunppsg.in/