mltest: Automatically test neural network models in one function call


So I got a lot of positive feedback on my last post on how to unit test machine learning code. A few people actually messaged me directly saying they caught a bug in their own code with the recommended tests, which is awesome! But these issues are still too common, and it is just as easy to forget to write a test as it is to write the bug in the first place. We need a better, more automated solution.

That is why we are introducing mltest: Automated ML testing in one function call.

Check it out!

Done. With incredibly little setup, we now are testing against several different common machine learning issues.

To install it, just run:

The function call mltest.test_suite(…) is the main powerhouse of this library. It runs several tests including:

1. Variables change

2. Variables DON’T change

3. Logits Range

You can also set the range to check to be whatever you want. Let’s say you expect to have a tanh on the logits, you can set a check to make sure all of the values of the logits are in (-1, 1).

4. Input dependencies

Of course, any of these tests can be turned off manually with flags in test_suite(). See the code for documentation on how to do this.

mltest setup

This call will automatically reset the default tensorflow graph and set tensorflow’s, numpy’s, and python’s random seeds. It’s very easy to forget to seed your random values, and can cause a massive headache when trying to recreate bugs.

This suite is still in beta, so if you have any requests or notice any bugs, please add an issue on Github! Also, I accept pull requests. ;)

Machine learning researcher in the making.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store