Hacker News Reader

Comment by nhgiang | Hacker News Reader

For instance, in traditional tests you may need two test cases for some behaviour. One is for usual inputs and one for some edge case.

With Hypothesis, you can combine those two tests into one, by letting Hypothesis generate the inputs for you. Hypothesis is pretty smart to find edge cases that you usually don't think of.

Overall, it reduce my time to write tests by at least half, maybe 75% even.

The caveats are: 1/ Tests may take visibly longer to run. 2/ If you have setup/teardown or fixtures that you want to repeat each generated input, you have to do this manually. This is imo the biggest weak point of Hypothesis right now.

View on HN

> Hypothesis is pretty smart to find edge cases that you usually don't think of.

This is worth reemphasizing.

For certain kinds of hard problems, I've found it really valuable for building confidence in an implementation.

If you happen to have a reference implementation, with a small surface API, it can be a very fast way to write tests: tell hypothesis to input anything it wants and validate that both libraries produce the same result. It's not magic, you will probably still have to give hints about how to build interesting inputs, but still a very quick way to get a lot of coverage. (Also be sure to crank up the number of examples to build more confidence, 100 is often not enough to really explore an input space).

I like pytest for the ergonomics of its static setup/teardown/fixtures (plus not having to stuff everything into a unittest.TestCase). Unless you mean using generated input in those bits? In which case yeah, not ideal. At $dayjob we do some chonky processing; generating new input for each property we're testing is just barely ok right now, but that's probably not going to hold for very long.

[1] https://github.com/pytest-dev/pytest-subtests

Sorry for not being clear. This is what I was talking about:

https://github.com/pytest-dev/pytest/issues/916

It is a known issue. Basically your fixture/setup/teardown, which you expect should run for every example generated by Hypothesis, actually only run once.

Oh yeah I remember that. Hasn't been an issue for me, thankfully.

Reply on news.ycombinator.com