# Unit testing with the Hypothesis library

# Introduction

Unit testing is a software testing method that allows any person involved in coding to validate that a given block of code performs as *expected*. With the traditional testing workflow, we need to specify precisely the inputs to test and therefore, if we choose too simple or easy inputs, then the test only covers those and isn’t a reliable tool to assess the quality of the code in case of a more complex or unexpected input.

The *Hypothesis* library allows us to describe **how** the inputs look like instead of specifying **what** they are. Then it generates many test cases with inputs corresponding to that description. The framework stops when it encounters a failed test case and raised an error specifying which input crashed the code.

# A trivial example

- Let’s write a function
*divide*and a simple test function*test_divide*.

The test case code runs successfully and it seems that our function works correctly.

def divide(x, y):

return x / ydef test_divide():

divide(1, 2)

- Now let’s rewrite the
*test_divide*function*Hypothesis*framework and call this new function*test_divide_hypothesis*:

from hypothesis import given, settings

from hypothesis import strategies as st@given(st.floats(), st.floats())

def test_divide_hypothesis(x, y):

divide(x, y)

The library will generate multiple inputs to test the code and in that case, it will report a *ZeroDivisionError *for the inputs (x = 0.0, y = 0.0), meaning we have to handle this case before going forward.

# How to use the Hypothesis library

Writing a *Hypothesis* test is pretty straight-forward :

- Decorate your test function with a decorator @given.
- Describe the input data with
**strategies**.*Hypothesis*has support for primitive data types and also provides the ability of building complex ones. A detailed reference can be found here. - Run the test with standard Python
*unittest*or*pytest*.

# Use case for Data Analytics / Machine Learning projects

*Hypothesis* supports numpy and pandas strategies.

To demonstrate this, we are going to test the function *scale_min_max_v1*.

`def scale_min_max_v1(df):`

for feature in df:

series = df[feature]

df[feature] = (series - series.min()) / (series.max() - series.min())

return df

Firstly we tell *Hypothesis* to produce a data frame with two columns “a” and “b” as float data type. We also add another decorator @*settings* to quickly rerun the failed test case later on.

def float_data_frame(features):

return data_frames(columns=[column(name=feature, dtype=np.float) for feature in features])@given(float_data_frame(['a', 'b']))

@settings(print_blob=True)

def test_scale_min_max(df):

out = scale_min_max_v1(df)

if not out.empty:

for feature in out:

series = out[feature]

assert series.isnull().any() == False

assert np.isfinite(series).all() == True

After running the test (*test_scale_min_max*), *Hypothesis* reports the following error (data frame with values of 0 for both columns “a” and “b”) :

`Falsifying example: test_scale_min_max(`

df= a b

0 0.0 0.0,

)

You can reproduce this example by temporarily adding @reproduce_failure('5.16.0', b'AXicY2TAAQAAMgAC') as a decorator on your test case

We handle this error by returning 0 in the case of a 0 denominator. Let’s create a new function `*scale_min_max_v2*` to reflect those changes :

`def scale_min_max_v2(df):`

for feature in df:

series = df[feature]

denominator = series.max() - series.min()

if denominator == 0:

df[feature] = 0

else:

df[feature] = (series - series.min()) / (series.max() - series.min())

return df

With this updated function, we can check whether the failed test case is solved or not. To do that, the *Hypothesis* library gives us the decorator to add to *test_scale_min_max :*

`@reproduce_failure('5.16.0', b'AXicY2TAAQAAMgAC')`

This time, we receive a new message “*hypothesis.errors.DidNotReproduce: Expected the test to raise an error, but it completed successfully.*” which confirms the correctness of our fix!

Now let’s remove the *reproduce_failure* decorator and rerun the test. A new error is discovered with an infinity value (np.inf) for the column “b”.

Falsifying example: test_scale_min_max(

df= a b

0 0.0 inf,

)You can reproduce this example by temporarily adding @reproduce_failure('5.16.0', b'AXicY2RABf8/QBkAEKkB8Q==') as a decorator on your test case

Similarly, we implement our new fix in a new function *scale_min_max_v3* which replaces infinity values with 1e18 or -1e18.

`def scale_min_max_v3(df):`

df.replace(np.inf, 1e18, inplace=True)

df.replace(-np.inf, -1e18, inplace=True)

for feature in df:

series = df[feature]

denominator = series.max() - series.min()

if denominator == 0:

df[feature] = 0

else:

df[feature] = (series - series.min()) / (series.max() - series.min())

return df

After rerunning the test, another error is raised by the library (np.nan values in the column “b”).

Falsifying example: test_scale_min_max(

df= a b

0 0.0 NaN,

)You can reproduce this example by temporarily adding @reproduce_failure('5.16.0', b'AXicY2RABf8/gClGBgAQqwHy') as a decorator on your test case

Let’s fix the error in *scale_min_max_v4*.

`def scale_min_max_v4(df):`

df.replace(np.inf, 1e18, inplace=True)

df.replace(-np.inf, -1e18, inplace=True)

df.replace(np.nan, 0, inplace=True)

for feature in df:

series = df[feature]

denominator = series.max() - series.min()

if denominator == 0:

df[feature] = 0

else:

df[feature] = (series - series.min()) / (series.max() - series.min())

return df

After removing the decorator *reproduce_failure*, the *Hypothesis *library has no more errors to report!

By default, *Hypothesis* generates 100 example test cases for each @given decorator (this can be increased with *@settings(max_examples))*.

# Conclusion

The *Hypothesis* library helps us to discover hidden bugs in our code by automatically generate multiple and complex test cases (see testimonials here). As a result, it’s a little bit more time spent during the development phase but so much time saved if we had have to identify the root cause of a bug in production with unexpected inputs. Happy unit-testing!