Clang is better than GCC
yurichev.org
yurichev.org
> But that bug in my code ruined one month of my work. I coded without realizing tests don't work at all. OUCH!
Sorry to say but that too is your fault. First thing you're supposed to do with tests is make them fail, then make them pass. The only safe way to handle it is that if you've never seen your test as written fail, then it never fails.
As an aside, the first line says:
> Sorry for the provocative title, but I'm too emotional these days.
And this really ticks me off for some reason, probably because the entire web and beyond has moved to that form of expression where one exaggerate and then say "yeah but I have my reason". I miss adult discourse.
And clang is really great at catching more issues than other compilers would, and then reporting them better than the others. So the title isn’t far off.
I don't strictly disagree with your first point, but I disagree with the attitude from many dev regarding tests that lead to it becoming too common like in this article (and I'm not piling on the guy, it's very very common).
It's like people have forgotten what tests are for and thus what you want from them: a test that pass is essentially useless, it completes the green check mark and that's it, it doesn't matter. It's the test that fails that provide you information, that matters to you.
One might be tempted to disagree, tell me that passing test also provide information, by ensuring this and that and I will insist that they take a second and realize that no, they don't. Ultimately, the only value they have is in fact that they can fail, it's always the failure that matter.
There are indeed tests which fails to fail for other unplanned reasons, by that's not really what I'm getting into, I'm aiming specifically for the attitude of tests for the sake of tests.
> It's like people have forgotten what tests are for and thus what you want from them: a test that pass is essentially useless, it completes the green check mark and that's it, it doesn't matter
This is, in my opinion, not true in some cases. Sometimes, I'll write unit tests that never fail, and are purely to allow me to verify one of my assumptions about the code. Basically, the unit test is a tool to force me to think through each line of the code, and that's it.
Typically, when I do the above, I also use a debugger and/or line coverage tool to ensure the test actually reaches part of the code I'm trying to reason about, and with the debugger, this sort of "test-writing as a tool to think about code" is in my opinion much more powerful than just reading code. I do this a lot in code review when I don't understand someone's code, since it's easier to run sample values through it and set breakpoints in a test, than it is to do all that in my head looking at the diff view.
Now, you might "no true scottsman" argue that those are not tests (and indeed, I often don't bother to check them in), but I think they are.
I also think integration and blackbox tests that never fail are fine, and expected. If I add a website uptime monitor (a "foo.com returns 200" type test), should I immediately take my site offline for a while to make sure it fails?
Ideally yes, to test the uptime monitor!
That is fine and has value, but that is usually called an assertion (and most language have them under some sort of assert() function), which is different from an unit test.
> If I add a website uptime monitor (a "foo.com returns 200" type test), should I immediately take my site offline for a while to make sure it fails?
How do you know if it works otherwise ? In fact for that specific exemple, the most naive version of that test actually misses many failure cases as we've seen again and again in the last decade with status page that fail to detect any error
unit tests let me pick sample values, and I do what I'm talking about using unit test libraries, not assertion libraries. Perhaps you misunderstood me.
> many failure cases as we've seen again and again in the last decade with status page that fail to detect any error
That is a political problem, not a testing problem. The majority of status pages that fail to detect errors are because the company decided to have a human update it because the test failed too often, or did not differentiate "error" and "warn" or provide a human description.
Status page uptime is often a political matter for a company, and no unit testing practices will fix a political problem.
But there are many subtle reasons why a test might not get written. Sometimes it’s deadlines. Sometimes it’s because a component is hard to test in isolation. Sometimes it’s just an honest mistake.
So, while I would encourage everyone to write tests I also won’t judge them for not doing it.
But it's also important to remember why you write test: for their failure cases, not their success.
This code will not pass a code review in my last shop:
1. First get the return code: rc = memcmp(…
2. Then compare, and please put the constant first: if( 0!=rc ) …
C and C++ are dangerous tools, you should not use in production all the “sugar” they provide.
> And this really ticks me off for some reason, probably because the entire web and beyond has moved to that form of expression where one exaggerate and then say "yeah but I have my reason". I miss adult discourse.
I completely agree. I've seen a lot of people try to excuse being rude to people because they're extra emotional at the moment. If you're extra emotoinal at the moment, *don't write it*. It's more reasonable to use as a post-facto reason. (Example: Sorry. Yesterday I wasn't in my right mind and I said some mean things to you that I regret saying. It won't happen again.) But it's not at all okay to try to excuse what you're about to write. That's not an apology at all. That's like a person with anger management problems trying to excuse being able to hit people whenever they want.
This was always adult discourse.
We just have rose colored glasses to think otherwise.
They did not say they didn't do that, you're assuming.
If I were the person here, I might write a simple test with 'val1="foo"; val2="bar";' to make it fail. That would make it fail because the first byte is different, while I would think it failed because the entire value is different.
You might say that I should have also tried "bar" and "baz" (which would reveal the issue), but it's infeasible to try all possible inputs.
Most code is not written with exactly an excess of time to follow perfect practices every time, and so we must make tradeoffs.
In the article, the author confirms his test was not working:
> But that bug in my code ruined one month of my work. I coded without realizing tests don't work at all. OUCH!
The issue is exactly what parent mentions, that OP did not let his test fail first so that they were never confident in that the test worked.
This is absolutely the wrong way to do testing and the crux of what I was getting at: you should not change a single line of code within the test itself when making them pass or fail.
If you change the test itself, then it's not the same test, you have not ensured your test works, only that another test works.
I'm sorry if this sounds harsh or if my phrasing is not correct, but it's important to remember this. If "testing your test" implies "making modification to the test", then you're not doing it right.
Using Clang instead of GCC won't make him a better engineer.
> And this really ticks me off for some reason, probably because the entire web and beyond has moved to that form of expression where one exaggerate and then say "yeah but I have my reason". I miss adult discourse.
I'm on the same boat with you. Worse, people writing these titles also hold to their provocative titles firmly, making discourse almost impossible. Trying to read this loaded articles from title to last dot is exhausting and counterproductive, too.
This is why I also don't publish anything unless I can write a neutral title, and distill the post's tone to a neutral point.
LLVM comes bundled with a tool that will show you all the errors/warnings in a nested, hierarchical fashion
Along with being color-coded by whether they're enabled by default, and if they only exist for GCC compatibility.
You can display this by typing:
$ diagtool tree [optional-warning-level-like-Wall]
Here's a GIST where I collected some of the more obscure options you may be interested in adding:
https://gist.github.com/GavinRay97/bae1d93925e55ce0a0084946f...
The real problem here is not the bug in the memcmp() call, it's that you should not use a memory compare to compare two structs in a test at all.
If the structs have any padding, you are invoking undefined behavior if the padding is not initialized.
And as we've seen, memcmp() opens the door to all kinds of errors.
Don't rely on a compiler to catch this for you. Instead write the code correctly in the first place.
The author's struct has x, y, and z fields, and the right way to compare these structs is to explicitly compare these three fields.
For example, based on the author's code (using a typedef for the struct, and naming it xyz instead of tmp):
typedef int bool;
typedef struct {
int x, y, z;
} xyz;
bool xyz_equal( xyz one, xyz two ) {
return(
one.x == two.x &&
one.y == two.y &&
one.z == two.z
);
}
I agree whole-heartedly.
There is no situation in which `memcmp` should be used with structs. In code review this would have been flagged as an error, no matter how much the author cried "but but but ... there's no padding!".
Even if there is no padding right now (compiler flags, or struct has perfectly aligned fields), at some point in the future this code will be reused in a spot where there will be padding (compiler upgrade, someone adds a new field at the beginning of the struct, etc).
It's the same with serialising structs, you have to do it field by field.
We were all talking about better ways to use memcmp(), such as whether to use the "!= 0" or not, or my memeq() wrapper.
Those are all good things to consider if memcmp() is the right solution in the first place.
I only had the "aha" moment after I took a break for a long walk and dinner.
I should do that more often before assuming I have the right answer... :-)
Maybe this is an example of what they call an XY Problem:
https://news.ycombinator.com/item?id=33762154
https://twitter.com/jfbastien/status/1592578174763745280?ref...
In general everything from the string.h header should be only used on non strings with great care.
Another variant is having x as an int. It is supposed to truncate the int on assignement, but instead it simply dumps the high bits in the alignement.
As the saying goes: Working code > crashing code > silent corruption.
But still, it's good having both clang and GCC. The competition makes both better. If there was only one, it would stagnate.
Clang much less so. But clang supports all the modern “important” isas, so as a practical matter, it doesn’t matter that much for everyday use.
Of course, 99.9% of people don't use any obscure architecture, so that's irrelevant to them.
Or at least stop jamming so much stuff into one line, and use whitespace around operators!
To be fair to C, problems like this exist in a lot of languages, even more-strongly-typed ones. At some point, a linter comes in handy. It's nice if the compiler has linting built in, but at some point you need to draw a line between "compiler warnings/notes" and "lint to be found by an external tool".
cppcheck picks this up (a bit cryptically):
clang-gcc.c:17:72: error: Invalid memcmp() argument nr 3. A non-boolean value is required. [invalidFunctionArgBool]
if (memcmp(m_result_original, m_result_my_version, sizeof(struct tmp)!=0))
<source>:17:1: error: V526 The 'memcmp' function returns 0 if corresponding buffers are equal. Consider examining the condition for mistakes.
The error here is in the programmer relying on one tool as source of truth.
struct tmp* m_result_original=...;
struct tmp* m_result_my_version=...;
... memcmp(&m_result_original, &m_result_my_version, ...)
Ironically, GCC 12.2 warns about this error while clang doesn't :)
The warning message produced by GCC is "'memcmp(...)' specified bound 12 exceeds source size 8 [-Wstringop-overread]", and it is printed even without -Wall, but only when the third argument is `sizeof(struct tmp)` rather than `sizeof(struct tmp)!=0`.
But as @Stratoscope pointed out, memcmp() is still problematic if the struct has padding.
Comparing it to false requires me to know that false is 0. Comparing it to a ptr requires me to know that 0 represents a pointer pointing to nothing.
C does not have a boolean type. (The `true` and `false` defined in stdbool.h are macros expanding to the integer literals 1 and 0, respectively.)
Functions which return 0 on success I always compare with 0 as “if (foo()) error();” easily gets confusing. If I see “if (foo() != 0) error();” then I can easily tell what’s going on because C is funny like that.
Conversely, if I’m testing for truthfulness or NULL I’ll use the boolean operators.
Also, put the constant value (0 or true or null) on the left side.
1. There are similar functions (like std::lexicographical_compare() in C++) returning bool instead an int.
2. It's not crazy to read memcmp() as comparing for equality rather than ordering. i.e. I think when you see 'if (memcmp(a, b, n))', it's quite natural for one's brain to read it as "if a and b are equal..."
For someone who uses memcmp() on a daily basis, you probably don't encounter these misconceptions. But as someone who writes C++ and merely deals with C once in a while in the process, both of these possibilities have tripped me up multiple times when seeing if (memcmp(...)). Writing != 0 has always cleared that confusion for me, since it immediately signals the result (a) isn't meant to be read as a Boolean, and (b) might be negative, thus putting my brain on alert.
int memeq( const void* lhs, const void* rhs, size_t count ) {
return memcmp( lhs, rhs, count ) == 0;
}
I would also use a typedef for the struct, so you don't have to keep writing "struct" everywhere:
typedef struct {
int x, y, z;
} tmp;
if( ! memeq( m_result_original, m_result_my_version, sizeof(tmp) ) )
That said, this code, even with my updates, was wrong all along. Using memcmp() on two structs, even with a wrapper, is a bad idea. What if there is padding?!
This particular struct will not have padding, but in the more general case you shouldn't count on that.
What you really need to do is have a function that explicitly compares the x, y, and z fields of these two structs.
Then you won't have to worry about uninitialized padding, you will never get the length wrong, and everything will always just work.
if (!((my_ptr = malloc (sizeof (struct my_struct))) != NULL))
exit (1);GCC is still the king of the hill though - the amount of support for various ISAs, new features like C++20 stuff - it's far from lagging clang in hugely significant ways. (GCC user since 2.95 on Solaris/SPARC!)
I know it used to. No idea if it’s still the case.
Would be cool to hear more about this.
I guess it isn't as bad as some other compilers still catching up with ISO C++14 and C++17 in embedded space and classical UNIX.
clang++ -std=c++2b --gcc-toolchain=/usr/local/gcc-13.0.0-dev
The C++ compiler and the C++ standard library exist as separate components. You're free to mix and match them. You just need to configure the relevant compiler and linker flags (which may be harder on GCC than Clang, for instance)
And if you mean to just #include <ranges>, that's not right at all
For instance, installing LLVM/Clang on Fedora Rawhide uses Redhat GCC 12.2 as the default stdlib right now, and including <ranges> works perfectly fine:
[user@MSI ~]$ clang++-15 -v
clang version 15.0.0 (Fedora 15.0.0-5.fc38)
Target: x86_64-redhat-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-redhat-linux/12
Selected GCC installation: /usr/bin/../lib/gcc/x86_64-redhat-linux/12
this is not really true. Aside from the fact that the standard library need not be a library at all and could just be built-in, in practice the standard library provided by a compiler is intimately tied with that (specific version of the) compiler. libstdc++ uses plenty of GCC specific non-standard extensions and relies on implementation specific and often undocumented behaviour.
Clang tries to implement all the same extensions and GCC developers try not to break clang support too often (sometimes even providing clang specific workarounds or gating out non-implemented features), but as you have experienced, breakage does happen form time to time.
Both compilers are good enough. It's more about style.
1. No need to save spaces, your fingers won't fall off from pressing the buttons. 2. Pairs of brackets are not so much for the compiler (because they can be freely nested inside each other), but for people. 3. The inverse notation of the condition saved embedded programmers from such errors, and it saves, and will save:
if ( 0 != memcmp( blah-blah-blah ) )
It's very old school culture, and it works.
While this is a different than warning you about errors it still is an important aspect of compiler.
Having used both compilers quite a bit they each have their own strengths. Although I will add clangs code generation as gotten a lot better over the years that it does not generate crazy bad machine code as often anymore, but I still see it happen.
Wasting weeks on this bug is on the author
malloc, calloc, realloc, memcpy, memset, memcmp, strcpy, strncpy, and a few others. They're a rich source bugs, usually:
1. an off-by-one error due to the terminating 0
2. an overflow vulnerability (particularly with malloc/calloc/realloc)
> The problem is that (sizeof(struct ...)!=0)==1, so size=1 always for memcmp(). Instead of comparing two tmp, my code compared only 1 byte of each structs.
This isn't just due to the type coercion of the third argument to memcmp though, is it? It's at least as much due to the fact that the _return value_ of memcmp is a valid `if` condition. I know that "zero is false, everything else is true" for integers is just as much a "feature" of C as the coercion of the comparison to a size_t, but I feel like it's less obviously a useful feature, given how easily it is to just manually write `!= 0` if you happen to want the existing behavior. Are there any compiler warnings or hints for being able to disallow this specifically?
Yep! I'm thinking that it might be possible to approach it from the side of how the expression is constructed rather than what it actually yields when evaluated though. I'm not positive, but my instinct is that it should at least be technically possible to statically detect whether or not a condition was constructed from a set of recursive rules like this:
1. `true` or `false` from std.bool.h
2. a variable or struct field of type `bool` from stdbool.h
3. a comparison expression using either equality or inequality operators
4. some composition of instances of these rules with boolean operators
5. a function call where all returned values are instances of these rules (applied transitively across nested function calls)
You catch a lot of these errors and UB. It will also force you to code more idiomatically as you grow the project.
> Or maybe I should stop using pure C...
Yes, you should.
if (*m_result_original != *m_result_my_version)
Isn't that case that unit tests were a waste of time? :P
* He doesn't say how much time he actually spent, maybe he spent a whole week writing his (poor) unit tests. Still, that part was the waste of time?
How much time was wasted because of the badly written test is anyones guess.
export CC_LD=lld
CC=clang meson setup -Dbuildtype=debug -Dwarning_level=3 build-debug-clang
CC=gcc meson setup -Dbuildtype=debug -Dwarning_level=3 build-debug-gcc
CC=clang meson setup -Dbuildtype=debugoptimized -Dwarning_level=3 -Db_lundef=false -Db_sanitize=address build-asan
ninja -C <name of build folder>
CC=<whatever> bazel buildGood luck trying C++20 with clang.