Stop Writing Tests!

Zac Hatfield-Dodds (~Zac-HD)




We often think of manual testing as slower and less effective than automated testing, but most test suites haven't automated that much! Computers can execute all our pre-defined tests very quickly - and this is definitely a good thing, especially for regression tests - but the tricky parts are still done by humans. We select test cases (inputs) and check that the corresponding outputs make sense; we write functions that "arrange, act, and assert" for our tests; and we decide - or script via CI systems - which tests to execute and when.

So lets explore some next-generation tools that we could use to automate these remaining parts of a testing workflow!

PROPERTY-BASED TESTING helps you to write more powerful tests by automating selection of test cases: instead of listing input-output pairs, you describe the kind of data you want and write a test that passes for all X.... We'll see a live demo, and learn something about Python in the process!

CODE INTROSPECTION, and a handy templating tool, can help write tests for you. Do you need to know any more than which code to test, and what properties should hold?

ADAPTIVE FUZZING tools take CI to its logical conclusion: instead of running a fixed set of tests on each push, they sit on a server and run tests full-time... fine-tuning themselves to find bugs in your project and pulling each new commit as it lands!

By the end of this talk, you'll know what these three kinds of tools can do - and how to get started with automating the rest of your testing tomorrow.


Participants should be comfortable using a standard testing library like pytest or unittest, and interested in taking their testing to the next level! The talk is suitable for both experienced users of hypothesis, and Pythonistas who have never even heard of it before.

Content URLs:

Speaker Info:

I'm a researcher at the Australian National University, and maintain a number of Python packages - mostly focussed on testing. When not at my desk grumbling about the buggy state of software, I can be found with a book, a block of chocolate, and a plan to go somewhere out of phone range for a walk.

Speaker Links:

Section: Developer tools and automation
Type: Talks
Target Audience: Intermediate
Last Updated: