MinUnit: a minimal unit testing framework for C
jera.com
jera.com
When the debugger is not attached, it provides additional incentive to fix the tests ;).
(Hanging CI on failed tests on remote servers is, admittedly, annoying at best. MinUnit will also only report a single failed test in that case, I note.)
https://docs.python.org/3/library/ctypes.html
but your unit test is written in Python, including all the setup/teardown stuff and asserts. And you run it using the stock Python test runner (or some wrapper around that):
https://docs.python.org/3/library/unittest.html
(In theory, this all can be done even smoother if you use Cython, because that lets you interface with C more naturally than ctypes. I haven't tried that, though.)
Note that this all makes sense for C APIs, not C++ ones. With C++ APIs, as soon as classes etc come into the picture, and especially once you start dealing with STL and Boost data types, it becomes too difficult to invoke that stuff from Python. However, the implementation can be in any language - what matters is what the tested API looks like.
Most of the students would waste most of their time in lab sessions adding and removing printf statements to one huge `main` function, and trawling through the resulting mess on the console.
I was quite satisfying to guide them through the process of:
- Articulating exactly what they're looking for in the console (e.g. "these numbers should add up to the same as that")
- Have them write that down in the code (e.g. `mu_assert("results add up", x + y == z);`)
- Remove the corresponding printf calls
Due to sheer amount of output from printf, many students had cobbled together elaborate nested-if-based one-char-at-a-time scanf-parser menu systems to choose which parts of the program to run with which inputs; the results being far more complicated than the actual programming assignments they'd been given! Since tests only output when they fail, all of this complication could be thrown away.
Then, the magic parts:
- Leave the tests in place, so they'll be checked every time
- Keep adding more tests to narrow down a problem more specifically
- Wrap the tests in `for (int i = 0; i < 1000000; i++) { ... }` to check far more cases than would be feasible manually
Besides driving home the idea of automation, testing across many inputs in a loop usually made debugging easier. Due to the effort of setting up, checking and tearing out printf debugging statements, many students had skipped checking trivial cases like 0 and 1, and instead were trying to wade through the output for an input like 50. Adding a loop caused those trivial cases to be checked, and the problems became obvious :)
I've used both, although I wouldn't say I've used any of the advanced features of Check. Check seems to be a bit more maintained and documented, but I don't remember having any particular problems with Cunit the last time I used it.
Check's documentation also includes a list of the other C unit testing frameworks its developers know about[2]. I'm curious to check out AceUnit, which is said to be usable in an embedded context.
My approach for testing embedded functionality has been to shim any hardware-specific stuff in the code I want to test and run the tests using Check on the machine hosting the build. This "works", but it misses errors that occur because of differences in e.g. the size of size_t. Mind you, I've only been doing embedded for fun recently.
[0] https://libcheck.github.io/check/ [1] http://cunit.sourceforge.net/ [2] https://libcheck.github.io/check/doc/check_html/check_2.html...
ETA: Disappearing from the internet. Won't be around for discussion. Apologies for the post-and-run.
What you need is something more like this. (I'll write out one case, and you can figure out the rest.) It's more lines, but I don't think you'll feel like it's bad value per LOC.
For the actual condition you write in your code, you want a macro that expands to something with __FILE__ and __LINE__, and uses the stringizing operator to get the LHS and RHS as strings. Here's one that compares to ints (hence II) for equality (hence EQ).
#define TEST_EQ_II(A,B) (DoTestEqII((A),#A,(B),#B,__FILE__,__LINE__))
void DoTestEqII(int64_t a,const char *a_str,
int64_t b,const char *b_str,
const char *file,int64_t line)
{
DoTestII(a,a_str,b,b_str,file,line,a==b,"==");
}
void DoTestII(int64_t a,const char *a_str,
int64_t b,const char *b_str,
const char *file,int64_t line,
int result,const char *oper)
{
if(!result) {
Print("%s%s%" PRId64 "%s: test failed\n",file,LINE_PREFIX,line,LINE_SUFFIX);
Print(" Expected expression: %s\n",a_str);
Print(" Operator: %s\n",oper);
Print(" Actual expression: %s\n",b_str);
Print(" Expected value: %" PRId64 " (0x%" PRIx64 ")\n",a,a);
Print(" Actual value: %" PRId64 " (0x%" PRIx64 ")\n",b,b);
Break();
}
}
#define Break() __debugbreak()
#define LINE_PREFIX "("
#define LINE_SUFFIX ")"
For POSIX-type stuff, e.g., Emacs:
#define Break() asm volatile ("int $3\n") /* if you get a build error here, your computer is shit. */\
/* Throw it away, and buy a proper one, with an x64 CPU. */
#define LINE_PREFIX ":"
#define LINE_SUFFIX ""
(If you use vim, or whatever... I'm sorry. But lesser editors often understand the standard FILE:LINE: markup, so even when you're slumming it, you're not, like, totally slumming it.)
You can probably figure out how this extends from int equals int to int not equals int, int less than int, blah blah... and from there to string vs string, and double vs double, and so on. It's not really the operations that are the key thing, because of course they're easy to add; for my money the benefit comes from auto-generating the output text from the condition, and from having it integrate tidily with the calling process - Visual Studio, Emacs, other (notepad, vim, etc.) - in such a way that you can jump immediately to the source location when a test fails.
Where the article and I do agree: the conclusion, all of it. Writing a unit test system isn't hard, and writing your own isn't crazy. It probably won't take you even one hour.
P.S. if you like C++, you can go one better, and have the output text auto-generated without even having to say what sort of comparison it is: https://github.com/philsquared/Catch
Only downside is that while you'll save one hour not writing your own, you'll lose ten hours waiting for it to compile. Also, good luck trying to write test programs that have command line options. This is what I believe they call "technical debt".
I'd love to see the scripts you used to manage your suite if you wouldn't mind sharing.
future languages should have testing as the method definitions (i'd call it first class citizen testing)