Unit Testing for Pyspark

animenon


Description:

Unit testing for pyspark code.

  • Using pytest for testing PySpark code
  • Utilising mocks and pytest-spark
  • Testing python UDF

Prerequisites:

  • Knowledge of Python and PySpark
  • Knowledge of unit testing

Speaker Info:

Anirudh, Working as a Sr. AI Engineer at a MNC.

Have experience working on data engineering and sciences.

Speaker Links:

I have spoken at BangPyper meetup earlier and also hosted Pune Stage at PyCon India last year.

Section: Data Science, Machine Learning and AI
Type: Workshop
Target Audience: Advanced
Last Updated: