and discussion: https://news.ycombinator.com/item?id=9112930
11,689 karma · joined June 13, 2008
Email: At gmail, with the account name vokes DOT s. My name is Scott, and I cook a mean frittata.
http://github.com/silentbicycle http://twitter.com/silentbicycle
Declarative real time programming NOW!
http://hackernewsers.com/users/silentbicycle.html
and discussion: https://news.ycombinator.com/item?id=9112930
This post (http://modelviewculture.com/pieces/it-s-not-you-it-s-the-sys...) mentions many ways well-intentioned projects can miss the mark, because of these kinds of issues.
As I stressed in the preceding post (http://spin.atomicobject.com/2014/05/16/radio-system-from-sc...), I'm intentionally doing these things the hard way, so I can learn how more of these things work at a deeper level than "use built-in functionality, configure to run at this baud rate, done". Otherwise, yeah, I'd use a 32-bit ARM chip and a better SMD transceiver. :)
I realized yesterday that it can also be used as the basis for a linear-time sorting algorithm: https://gist.github.com/silentbicycle/8389129
Benchmarking indicates that it's probably not competitive speed-wise compared to counting sort (of which it is a variant), but the implementation should be pretty easy to understand. It may be good for pedagogical purposes.
SUITE_LIST(suites) {
suite1,
suite2,
// ...
};
GREATEST_MAIN(suites);
but, again, I don't know if that's necessarily an improvement. RUN_SUITE is encapsulating saving the suite's name, optionally only running suites with names that match a certain pattern, and some other features that become tricky to implement with just a statically allocated function pointer array. I'm also trying to avoid doing anything "clever" with macros.(I work with most of the maintainers of those tools, and have some minor commits myself.)
Sloppy embedded code can kill people, or at the very least lead to incredibly expensive hardware recalls. With the exception of poorly thought-out privacy policies, it's unlikely that errors in a web app can cause that level of problems.
It's also much easier to deploy an update to a web app than a Mars rover or a car's brake controllers.
I use GREATEST_MAIN_BEGIN and GREATEST_MAIN_END so that people can put other things in main, around them. Also, there are often several RUN_SUITE calls, and I can't depend on vararg macros portably without also depending on C99. While there are some ways to work around that, they don't deliver enough value to justify the added complication.
One of my open issues is to add an ASSERT_EQ_MEMCMP function that does print out the difference, probably as a hexdump. Similarly, greatest could have something like 'ASSERT_EQ_NUM' for the cases where printing the value is meaningful, and an ASSERT_EQ_STRUCT that takes cmp and print functions. I've been thinking about it for a while, because I want to get it right before I change the API.
If I'm using fuzzing to run thousands of tests, and all the ones whose seed led to the same specific bit pattern are failing, that's particularly useful information. I found a particularly subtle bug in heatshrink (https://github.com/atomicobject/heatshrink) that way, but a single failing test would have looked like noise.
The single biggest difference is that greatest has more control over about which test(s) it runs. tinytest just bails at the first failure, whereas greatest can run everything and report, or individual test(s) or groups of tests whose names match a substring. That adds some to the implementation, but I've found it particularly valuable once a program has several modules and hundreds of tests.
Also, greatest supports parametric testing, which means that adding fuzzing/randomized testing is pretty straightforward (though usually project-specific).
tinytest requires C99, greatest doesn't. That shouldn't matter, but unfortunately sometimes it does. Embedded compilers for proprietary architectures can be lacking, in particular.
greatest also doesn't assume ANSI escape sequences are okay. Sometimes dumping out a bunch of [1;31m stuff is annoying. That's a matter of taste, though, and it'd be easy to make it optional.
I've tended to use CMock (https://github.com/ThrowTheSwitch/CMock) for mocking, though it depends on some Ruby-based tooling. The last time I looked into replacing CMock I wound up hip-deep in the binutils. (It was fun when I stopped getting google results from StackOverflow and started getting them from Phrack, though. ;) )
My preference is to have separate test_module_name.c files. Failing that, I would include tests in the module's .c file rather than the header, so that the header can be relatively short and focused on documenting the module's interface, rather than its implementation details.
greatest doesn't force any specific style there, by design.
I wrote a blog post about it a couple months ago: http://spin.atomicobject.com/2013/07/31/greatest-c-testing-e...
My primary goal with greatest is to assume as little as possible, and avoid imposing any additional constraints on projects just because people want to test them. I also want the implementation to be very transparent -- it's just a 600-ish line header file, carefully documented. It's a major pet peeve of mine when I have to work around features that try to automatically be helpful, but have edge cases that break down in opaque ways. (Build systems tend to be particularly bad in that way.)
Also, I'm beyond shocked that the name wasn't already taken. ;)
One of my upcoming tasks is making stdio optional in greatest, because printf doesn't do you a whole lot of good if you're running it on an Arduino or something, but in those contexts automated testing is especially valuable.