Let's Hunt a Memory Leak

sanketplus | 

Description:

Description

Python being a high level interpreted language, it never bothers us to deal with garbage collection. However, because we do not explicitly free memory like we do in C language, python has to clean it for us and hence there is garbage collection involved with python. While garbage collection works most of the times and never becomes an issue, when there is a memory leak in the code, we have no choice but to dig deeper to uncover it.

In this talk we will first look at how python manages memory and how does garbage collection works with python. We will look at some hands on examples to confirm the python memory management behavior and we will see why is it important to be aware about this behavior.

Further, we will look at a simulated memory leak scenario which we faced in production with Flask environment and we will look at one of the ways we used to hunt it down.

Outline

  1. Introduce Python Memory Management
    • look into what happens when an object is allocated and deallocated
  2. Look at an example to understand memory management behavior
  3. Introduction to the memory leak
    • and confirm the existence of leak
  4. Explore possible solutions
    • core dump analysis
    • tracemalloc
    • others

Takeaways

Understand python memory management and be able to debug memory related issues better.

TL;DR

Let's understand how python deals with memory and garbage collection, with that knowledge we will hunt a production grade memory leak!

Content URLs:

Blog entry: https://superuser.blog/detect-memory-leak-python/

Speaker Info:

Sanket is currently a Site Reliability Engineer with LinkedIn where he is working on Capacity Engineering initiative. He previously worked with Directi where he took care of Hadoop infrastructure, metrics, monitoring, and incident management pipelines. He is also into cycling and blogging.

Speaker Links:

Blog: https://superuser.blog/

Man page: http://sanket.superuser.blog

GitHub: https://github.com/sanketplus

Home: http://sanketplus.github.io

Previous talk: Serverless meets CI/CD, a hands on session at Large Scale Production Engineering Forum, Bangalore

Previous talk: Let's Build a Distributed File System at SRECon Asia, 2019 at Singapore

Section: Core Python
Type: Talks
Target Audience: Intermediate
Last Updated: