Python bindings to a distributed filesystem

Prashanth Pai (~prashanthpai)


7

Votes

Description:

GlusterFS is a distributed file system capable of scaling to several petabytes (actually, 72 brontobytes!) and handling thousands of clients simultaneously. It lets you build a scalable networked storage using pretty much any common hardware. It is easy to start small and add more storage later. GlusterFS combines blocks of storage over the network into a single, large storage pool that is served to client appliances. Gluster can deliver exceptional performance for diverse workloads. It is free and open source software enabling people like you to build data storage with high performance and great value, for a variety of demanding applications. In this talk, we aim to cover two major topics:

libgfapi-python (Python bindings to GlusterFS):

Python applications can now easily access petabytes of storage space present in your cluster. It's as easy as this:

from glusterfs import gfapi
v = gfapi.Volume(host, volume_name)
v.mount()

You can now perform I/O on the storage volume just as you'd do using Python's os module. For example:

v.listdir("/path")
v.mkdir("dir1")

glupy (Write translators using Python):

Translators are stackable modules that can intercept and/or transform filesystem I/O operations. glupy module will allow translator code to be written in Python. It's based on the standard Python embedding (not extending) techniques, plus a dash of the ctypes module. This talk will outline how one could leverage glupy to write custom translators.

While the python bindings allow the consumers to mount and use the filesystem, glupy will allow you to implement new features in GlusterFS internally.

Prerequisites:

  • Basic understanding of Python's os module usage.
  • Fundamentals of storage and filesystem

Content URLs:

Good to know:

Speaker Info:

Humble Chirammal works as Senior Software Engineer at Red Hat. He's been with Red Hat for more than 8 years. He is actively involved as a major contributor to GlusterFS open source project. His other interests include virtualization (QEMU/KVM, OpenStack) and emerging technolgies like containers (docker/kubernetes). He is also package maintainer for Fedora Linux distribution and a member of CentOS Storage Special Interest Group. You can reach him on:

Prashanth Pai works as Associate Software Engineer at Red Hat. He is primarily associated with and contributes to the open source projects like GlusterFS and OpenStack Swift. Find him on GitHub.

The speakers are co-maintainers of libgfapi-python project.

Section: Others
Type: Talks
Target Audience: Intermediate
Last Updated: