Description:

Abstract: In the rapidly evolving field of machine learning, ensuring the reliability and correctness of code is crucial. This talk will explore the importance of unit testing in machine learning workflows, detailing how Python can be leveraged to create robust test suites. We will discuss various types of testing, including assertions, functional testing with Pytest, and behavioral testing with Hypothesis, and provide practical examples to illustrate these concepts. By integrating thorough unit testing practices, machine learning practitioners can identify bugs early, validate their assumptions, and iterate faster with confidence.

Outline:

1. Introduction

   • Brief overview of the importance of unit testing in machine learning.
   • Goals of the talk: removing barriers to writing unit tests.

2. What is Unit Testing?

   • Definition and purpose of unit testing.
   • Testing a single unit of code in isolation.

3. Why Unit Testing?

   • Benefits: finding bugs early, detecting edge cases, checking assumptions, reducing surprises, and iterating faster.

4. Current State of Unit Testing

   • Common practices and tools used in the industry.
   • When to write unit tests in the machine learning workflow.

5. Types of Testing

   • Code testing for exploratory notebooks using assertions.
   • Functional testing with Pytest.
   • Behavioral testing with Hypothesis.
   • Examples of each type of testing and their applications in machine learning.

6. Assert Statements in Exploratory Notebooks

   • Using assert statements to validate data conditions.
   • Limitations of assert statements and why Pytest might be preferred.

7. Pytest Framework

   • Overview of the Pytest framework.
   • Pytest fixtures and parameterization for reducing code repetition.
   • Mocking dependencies to isolate code for testing.

8. Pre-Train and Post-Train Tests

   • Examples of pre-train tests: verifying feature generation pipelines, ensuring data quality.
   • Examples of post-train tests: validating model performance, checking for training-serving skew.

9. Common Issues in Machine Learning Pipelines

   • Potential problems in the feature generation pipeline, training tasks, model files, and serving pipelines.
   • How unit testing can help mitigate these issues.

Takeaways: - A clear understanding of the importance and benefits of unit testing in machine learning. - Practical knowledge of tools and techniques for implementing unit tests. - Strategies to integrate unit testing into existing machine learning workflows. - Insights from real-world examples to apply best practices in their projects.

Prerequisites:

This talk is aimed at data scientists, machine learning engineers, and developers who are involved in building and maintaining machine learning models. Attendees should have a basic understanding of machine learning concepts and some experience with Python.

Speaker Info:

Dhruv Nigam

Dhruv is a machine learning engineer who loves to build and deploy models at scale using Python. At Dream11, he leverage uplift modeling, reinforcement learning, and supervised learning to create action systems that enhance the user experience for over 100 million users. Before Dream11, Dhruv was a Director and founding Data scientist at Protium. He was key in scaling data science infrastructure from scratch to serve over 500k customers at Protium. He established core data engineering pipelines, data models, and deployment frameworks (GitLab CI/CD, Fast API, EC2, MlFlow) for machine learning models. He has spoken at various prestigious venues including a sponsor talk at CODS COMAD 2024. He has a bachelors and Masters in Electrical Engineering from IIT Bombay.

Ved Prakash

Ved is a skilled ML engineer with 9+ years of experience in conceptualizing and deploying large-scale machine learning and deep learning solutions. At Dream11, he has been a key player in reengineering the core contest generation engine. He is currently engaged in building state-of-the-art deep learning models tailored for tabular data domains. Before joining Dream11, Ved led the search and personalization initiatives at Paytm, where he built and deployed cutting-edge real-time machine learning solutions for 350 million users.

Speaker Links:

Dhruv

Linkedin - www.linkedin.com/in/dhruv-nigam-52531176.

Github - https://github.com/dhruvnigam93.

Twitter - https://twitter.com/druubeey.

Talk on credit risk modeling organized by Databuzz and DPhi - https://www.youtube.com/live/4acAw17khkY?si=vD-83gcY99CehXis.

Ved

https://github.com/ved93.

https://www.linkedin.com/in/vedthedataguy/.

Talk on real time ML- challenges and solutions - https://www.youtube.com/watch?v=DD5f-Gz1890.