Hacker News Reader

Comment by vortico | Hacker News Reader

For C and C++, what's wrong with using assert() in an integration_test() function or many <func>_test() functions, called when passed a --test flag or compiled as a separate binary?

For me, benchmarking isn't really useful unless it's a profiler in a real world use situation. Otherwise you have no idea how often each API function is called.

View on HN

For C++ I find both Google Test and Boost Unit Test Framework pretty usable. They both have some quirks but IMHO no show-stoppers. I prefer these to custom solutions because don't want to maintain yet another tool but to focus on my problem domain and it is nice when some fellow developers actually have experience with these tools.

The posted approach clocks in at 300 lines of C, and comes with the freedom to add whatever features you want. Testing is as fundamental as it gets; and unless you're aiming to write the One True Testing Framework, it's at most a days work to get one up and running. Google Test and Boost Unit try to hard to be everything for everyone, which makes them too complex; while failing to add anything that was missing from plain functions and asserts.

I prefer Boost.UnitTest to Google Test, which I also like. Boost.UnitTest has very nice sugar, but the big problem is the very slow compilation times.

That's why I like catch [0]

Super easy to use, header only so no need to compile anything beforehand, no external dependencies, reasonably fast compilation time, and does everything I need.

0: https://github.com/philsquared/Catch

Funny... I dumped Catch after a couple of days due to the terrible build times ;)

The fact it took over my programs' command line options and wouldn't give 'em back didn't help either. (Many of my tests are data-driven and use command line options to indicate where to read the data from - I already have a library to handle this! I didn't see - and still don't - why a test framework should be getting involved.)

I actually tried out Catch last night and dumped it almost immediately for the exact same reason. I was blown away by how slow it was compared to the rest of my (admittedly small) program in terms of build times.

I found using boost was a smaller hit to build times than Catch, and the suggestion of building catch in a separate translation unit didn't really help.

I'm guessing if your build times are already godawful you may not notice, but I sure as shit did.

Ultimate I found and went with dessert: https://github.com/r-lyeh/dessert

It's stupidly simple and doesn't do a lot of things that other C++ testing frameworks due, but it builds lightning quick and does 2 things I expect out of testing frameworks.

1. it tests, and 2. it reports failures.

I'll probably run into severe limitations that I can't deal with/work around, but for now I'm pretty happy with it and even if I end up using something else in 6 months, I'll still consider it worthwhile.

I think thats the main rub, think about the maintainers. Using a bespoke test framework, is often a way of introducing conflict, no matter how good a coder you are. As Jeff Atwood says, imagine the maintainer being a psychopath.. https://blog.codinghorror.com/coding-for-violent-psychopaths...

Would you rather have your psychopath stalkers tearing their hair out over the Boost dependency you dragged in? Once written it stays mostly the same, there is not much to maintain. I can't see why rolling your own framework would introduce conflicts; unless you're in an environment where writing code to solve problems is frowned upon, but then you have worse problems.

Nothing. That's the way I usually test my C code.

It has the drawback that one failing test aborts the whole program. The error messages are not the most informative, but it does the job.

Although hacky,

    assert(test_foo() /* Checks the sanity of Foo */);

does the job for the error messages, and replacing abort() with a custom function like `void check(int condition, const char *format, ...);` can make it not abort the program on an error.

I prefer the regular assert() though because I want the build to fail by an aborting test program, and all I care is the line number of failure. Another benefit is that abort() will pause gdb in the exact place that it crashed, with the stack intact if you want to inspect it.

You can also use the comma operator to document your assert()s, if you don't like using comments.

    assert(("test_foo() should be true", test_foo()));

https://en.wikipedia.org/wiki/Comma_operator#Asserts

I have used often and also seen in the wild the following form:

   assert(condition && "message");

This is easy enough to solve on Unix... simply fork() before each test. Just make sure that the master is single-threaded or this won't work.

Also an option is to siglongjmp out of the abort signal handler.

I've found it worthwhile to add a thin facade on top of assert to support out-of-band reporting of errors even if NDEBUG is disabled: https://github.com/codr4life/libc4/blob/master/src/c4/assert...

There's nothing wrong with assert, that's what I use for checking conditions as well. The framework posted is just a thin facade on top of regular functions and asserts to provide fixtures, dynamic grouping and benchmarks. Knowing how long a specific set of tests take to run for N repetitions is useful information. Since I wrote the test, I know exactly how many times the tested API is called. This is about spotting problems and trends early, once I find something interesting I'll drop down to profiling.

Reply on news.ycombinator.com