How To Log ML Experiments

Shagun Sodhani (~shagunsodhani)


Description:

One crucial and yet overlooked aspect of machine learning (ML) is logging. It is often narrowly interpreted as tracking the metrics (which is necessary but not sufficient). The talk will present a more general and broader interpretation - logs are an interface to an experiment. Logging is the mechanism for answering questions like "what" experiment was run, "why" and "how" it was run, etc. The talk will cover topics like the desiderata of a sound logging system and how logging helps make experiments reproducible. Some existing tools will also be compared, but the emphasis will be on raising awareness about high-level choices related to logging. The hope is that the audience will benefit by applying these suggestions in their ML workflows. The talk is useful for all ML practitioners. Anyone who designs, runs, and analyzes ML experiments should benefit from this talk.

By the end of the talk, the audience will gain a better understanding of the importance of logging in ML experiments. They would be able to incorporate the suggestions into their ML workflows and benefit from them. Logging is an under-appreciated aspect of ML engineering, and the talk will inform the audience about good logging practices.

Prerequisites:

There are no pre-requisites. Unlike usual ML talks that focus on specific frameworks, this talk focuses on generally-applicable insights (which are framework agnostic).

Speaker Info:

Hi! I am Shagun, a research engineer with Facebook AI Research. Before that, I was an MSc student at Mila (Quebec Artificial Intelligence Institute) with Prof Yoshua Bengio and Prof Jian Tang. I have also worked as a ML developer with the Data Science team at Adobe Systems.

My stack primarily comprises of Python (and related ML/DS/visualization toolkits). I love to play with new technology and look forward to meeting people at PyCon :).

Speaker Links:

Website: https://shagunsodhani.com

Previous Talks: https://shagunsodhani.com/talks/

Previous PyCon Talks:

PyCon India 2016 | Video

PyCon Canada 2018 | Video

Section: Data Science, Machine Learning and AI
Type: Talks
Target Audience: Intermediate
Last Updated: