Hacker News Reader

Comment by DyslexicAtheist | Hacker News Reader

I love the built-in static analyzer -fanalyzer option in gcc-10.

[1] https://gcc.gnu.org/onlinedocs/gcc/Static-Analyzer-Options.h...

Really? I tried to use it but I got a lot of false positives - in fact, every one of the errors in my medium-sized codebase was I believe a false positive. I spent a few hours last night looking at all of them and while I found a missing free, it was not one picked up by this analysis, but it happened to be in the same code that the analyzer flagged. Also the error messages are enormous in some cases.

It does show potential, and I hope it improves in future. Be nice to have a libre version of coverity one day.

Edit: Output from compiling nbdkit upstream with -fanalyzer: https://paste.centos.org/view/8381f926

Sadly, that's typical with static analyzers todays...

I like valgrind for this, combined with automated tests it gets decent coverage and finds actual (or at least probable) errors.

Valgrind is also prone to throwing up false positives. I find ASan much better.

Looking at the categories of warnings it produces, I'm not surprised. When I've used similar tools trying to detect the same things, I also saw false positives. It's probably not an easy set of problems, otherwise we'd have it built-in to more tools and enabled by default.

We routinely run nbdkit through Coverity and it finds bugs, although it too has false positives. Also the reports produced by Coverity are really nice - long enough to tell you where the bug is, but not too long to be overwhelming.

I've been meaning to formally prove one of our internal "mini libraries" using Frama-C. If we did that then no one would be able to complain about bugs in it :-)

Isn't it very similar to something Clang has had for years?

Tangential: I think many platforms that GCC supports and the kind of optimisation GCC provides might make it a superior choice for projects. For those developers its exciting. Also if an existing project is using it for a long-time and has not intentions of switching to Clang, for them too this would be interesting.

My project uses g++ but I’ve been using a clang static-analysis step in my CI pipeline for a while now.

Compiling with clang is much, much slower but it finds interesting errors sometimes. I only had to ifdef out one code section that it couldn’t understand (something about indexing an array with a constexpr function returning an enum class from a template parameter).

It depends on the project, but this one and another project I made work are both 100k lines of C++ and took half a day to setup. It was worth it imo.

Interesting that your build times are slower with clang. I've found on most of my projects clang is roughly 30% faster to compile in debug, but release I haven't seen a huge difference between the two.

Perhaps they are slower because Clang is doing static analysis in this case.

Static analysis is a lot slower but regular clang++ is also significantly slower than g++, like a 4-5 minute build time vs 2:30 on g++.

It might have something to do with a heavy usage of templates in a few files, I don't know.

Clang's autovectorization is better in a few functions I disassembled but otherwise the generated code doesn't benchmark any faster, it's better in some places, worse in others.

Nope, also slower for me.

My build times are identical between GCC and Clang (debug and release).

But TCC is 10x faster than both of them.

Clang has it although I've personally never tried it due to the horrible rigamarole of setting it up.

In GCC it's a compiler flag. In LLVM it's a convoluted process automated by either an irritating perl/python script (The python one isn't included by default for some reason despite being far more common these days) or AN ENTIRE WEB SERVER TO OUTPUT TEXT (Which thankfully isn't included by default).

I'm sure you could set it up in make but since my projects aren't large scale enough to really need it I can't be bothered, even if it's one of those things you'll probably only need to write once and can just copy and paste ad-infinitum.

Lest other people get the wrong impression from someone who's admitted not even trying it, running Clang's static analyzer is as simple as switching cc with scan-build. You can even drive it from clang-tidy. No Perl or Python setup in my experience.

You appear to have it installed somewhere. Open up scan-build in your favorite text editor if you don't believe me.

> Clang has it although I've personally never tried it due to the horrible rigamarole of setting it up.

    -% scan-build10 make
    scan-build: Using '/usr/local/llvm10/bin/clang-10' for static analysis
    ...
    scan-build: No bugs found.

What could ever have driven them to such a long and convoluted setup process?

Yep! Thanks for pointing out the perl script I mentioned. You appear to have completely ignored the rest of my post on them leaving important functionality to third party scripting languages.

Yes, to highlight that this "horrible rigmarole" and "irritation" you describe is entirely on the basis of there being a Perl script involved. Something I suspect the majority would overlook without even really noticing.

Though scan-build is usually the simpler option, clang itself does have an --analyze flag which writes analysis results in various formats, including the same html reports that scan-build would generate. But to see this on standard out

   clang --analyze --analyzer-output text ...

Will print the entire analysis tree in the same format as regular diagnostics.

The only problem is the CTU mess if you're analyzing more than one file, thus the aforementioned tools' necessity.

Hopefully in a future version the kinks are ironed out and we can just use the flag without any hassle. It's like if we needed to still manually link our files with ld before compiling them instead of clang auto doing it.

What do you love the most about the new analyzer pass? I haven't had a chance to try it out for myself yet but I'm looking forward to it.

An example to detect use-after-free: https://godbolt.org/z/zhiNLW

Basically this replicates what clang-tidy did

Too bad it doesn't detect the mismatch between new and free.

And if you fix it to use delete instead of free, you get "can't delete void *" errors. Perhaps not the best example code.

and if you fix the void->int, the warning goes away.

yeah that's why I changed int* to void* in the example, but I forgot to change new to malloc

I like not having to run another commercial tool[1] that will likely not be in use whenever I move on to the next project because no one has heard of it.

Biggest advantage I see is it's integrated into the compiler and so sees the same things the compiler does.

Having gcc do this out of the box helps people port their experience/skills with static analysis to other companies.

We already have clang-tidy and I like it too but it's nice to have a fall-back to compare when one produces a strange result. And a bit of competition is always good to have between such projects. And on most big projects it's not like you can just change the build system to use another compiler.

Also I found some interesting cases which valgrind didn't see because it was in an unreachable branch.

[1] https://news.ycombinator.com/item?id=22712338

Excited for this to come to gcc-arm-eabi-none

Imagine if -fanalyze was like rusts borrow checker

You Rust borrow checker requires special annotations and restrictions put on the code to do its job. I don't think you could something like that automatically on a C or C++ full codebase without having to manually annotate and refactor it somewhat. There are many common (and safe) C and C++ patterns that would be outright rejected by Rust's borrow checker, for instance initializing a structure or array partially if you're sure that nobody is going to use the initialized portion. Or having multiple mutable pointers/reference to the same object.

You could do something like that at runtime though, but then you have Valgrind, basically.

> Rust borrow checker requires special annotations and restrictions put on the code to do its job.

This is a good thing, because it makes lifetimes and ownership explicit and visible in the code. It serves the similar purpose as type annotations in function signatures.

> Or having multiple mutable pointers/reference to the same object

Sure you can have that with `unsafe`. And this is a good thing, because multiple mutable pointers to the same object is at best bad coding practice that leads to unsafe code, and you should avoid that in any language, including the ones with GC. Working with codebases where there are multiple distant things mutating shared stuff is a terrible experience.

If a C/C++ version of "borrow checker" could mark such (anti)patterns at least as warnings, that would bring a lot of value.

He wasn't criticizing Rust, he was just stating facts.

I read it differently, because he started with "You(r) Rust borrow checker", making his point automatically in oposition. But now after reading without this You at the beginning, I agree it was neutral.

My guess is that comment was made on a phone or tablet. It has a lot of small, autocorrect-looking mistakes. Other than the "You", for example there is one part where "initialized" is used where "uninitialized" is clearly intended.

I wish I could use that excuse, I just have the habit of posting first and then proofreading and editing, but noprocrast kicked in and then I switched to something else and now I can't edit it anymore.

However if I sounded like I wanted to belittle Rust I really expressed myself poorly, I love the language and hope it'll eventually become the new C++. If anything I was attempting to make the opposite point: you can't fix C/C++'s flaws with a smarter compiler. We're not two releases of GCC away from having safe C without having to change anything.

> Sure you can have that with `unsafe`

The parent was talking about the borrow checker so I only was talking about safe Rust code. Obviously if you consider that the entire C/C++ codebase is in a big unsafe {} block it'll work... because it won't do anything at all.

From Rust docs:

It's important to understand that unsafe doesn't turn off the borrow checker or disable any other of Rust's safety checks: if you use a reference in unsafe code, it will still be checked. The unsafe keyword only gives you access to these four features that are then not checked by the compiler for memory safety.

One of those four features is dereferencing pointers, and unlike references, pointers are not checked by the borrow checker. So you could bypass the borrow checker using unsafe code in a way, though most probably you should not.

Can you explain why multiple mutable pointers is bad practice?

I understand the benefits and the risks of them, and understand how Rust prevents both, but I dont yet understand why it's bad practice, and am interested to learn why.

Because it leads to hard to understand code.

If N unrelated (or loosely related) things can mutate the same object, then you get a O(x^N) explosion of potential mutation orders and in order to understand that, you need to understand all the (sometimes complex) time-relationships between these N objects. This gets even much worse when some of these objects are also pointed from M other objects...

On the flip side, in case of using a simple unique_ptr (or a similar concept), this trivially reduces to a single sequence of modifications.

Dont we in Rust still conceptually modify the vector multiple times, just through different means (usually a generational index or something)?

The one that affects you as a programmer most is Iterator invalidation. Iter borrows from the vector, you mutate the vector, iter blows up. Simple really. But a lot of code is like this. Borrow from hashmap, insert into hashmap, the slot gets moved around and your pointer is now invalid. That’s just vectors and hashmaps; imagine the possibilities in a much more complex data structure.

There are compiler optimisations you can do if compiler knows about aliasing, but that’s not so much a software authorship problem. There are some curly problems with passing aliased mutable pointers to a function written for non-aliased inputs, like memcpy and I imagine quite a lot of other code.

But common to all of these things is that it’s pretty hard to figure out if the programmer is the one who has to track the aliasing. In hashmap pointer invalidation, your code might work perfectly for years until some input comes along and finally triggers a shift or a reallocation at the exact right time. (I know this — I recently had to write a lot of C and these are the kinds of issues you get even after you implement some of Rust’s std APIs in C.)

I feel like this is something common c++ developers know, and it's not worth all the baggage to tag it. You just control the iterator inside the loop instead of in the loop declaration.

Is this still bad practice if the container can detect when this happens, like Java's? Not saying that it always has to throw like Java, I could imagine implementing a weak iter which we'd check before each operation.

> This is a good thing, because it makes lifetimes and ownership explicit and visible in the code.

No, it is additional burden. If it was possible to do it without annotations, you bet we would do it!

It's only a burden to the extent that type annotations are a burden. It's definitely possible (including in Rust) to do away with both, but that has downsides of its own.

I am not saying you cannot write code without them, but that you cannot do away with them without losing what they bring.

I would dispute that those common patterns are indeed safe, even if they could be argued they are when first written because code changes and can suddenly break your preconditions if they aren't enforced in the code itself.

C codebases then follow certain defensive programming customs to avoid reading uninitialized or out of bounds memory, at the cost of some performance. This is the right trade-off in C but, funnily enough, the more restrictive borrow checker has the opposite effect as you can give out inmutable and mutable references with wanton abandon because they get checked for unsafe behavior. It's the same difference as a a gun where the best practice is to keep it unchambered at all times to avoid the risk of a misfire, and a more modern gun with a safety: it's one more thing to think about but it actually smoothes the operation.

I'd describe the pattern in slightly different terms: when done right, restrictions in a programming language (or library/framework) are liberating for the programmer.

The restriction of immutability spares the programmer from worrying about whether unknown parts of the codebase are going to decide to mutate an object.

JavaScript's single-thread restriction (not counting web-workers) closes the door on all manner of nasty concurrent-programming problems that can arise in languages that promote overuse of threads. (Last I checked, NetBeans uses over 20 threads.)

Back to the example at hand, C has no restrictions, but that hobbles the programmer when it comes to reasoning about the way memory is handled in a program. It's completely free-form. Rust takes a more restrictive approach, and even enables automated reasoning. (Disclaimer: I don't know much about Rust.)

> for instance initializing a structure or array partially if you're sure that nobody is going to use the initialized portion. Or having multiple mutable pointers/reference to the same object.

Rust supports MaybeUninit<> for the former example, and unsafe raw pointers for the latter. It needs unsafe because these patterns are not safe in the general case and absent an actual proof of correctness embedded in the source code, a static analysis pass can only deal with the general case.

That's my point though, in both cases the developer needs to add additional syntax to make the intent clear. "Naive" Rust code that tries to do that stuff is rejected by the compiler.

I've expressed myself poorly in my original comment and apparently it looks like I was criticizing Rust but I wasn't. I was just pointing out that safety didn't come "for free" by toggling a compiler flag, you have to change the way you code some things. If C and C++ were to become safe languages, code would need to be rewritten using things like MaybeUninit, split_at_mut, RefCell etc...

While it isn't at Rust level, that doesn't stop Google and Microsoft from trying.

"Update on C++ Core Guidelines Lifetime Analysis. Gábor Horváth. CoreHard Spring 2019"

https://www.youtube.com/watch?v=EeEjgT4OJ3E

>You Rust borrow checker requires special annotations and restrictions put on the code to do its job. I don't think you could something like that automatically on a C or C++ full codebase without having to manually annotate and refactor it somewhat.

What about with a constrained (not necessarily general purpose) AI with the expertise of Scott Meyers, Andrei Alexandrescu, Herb Sutter and Alexander Stepanov?

I think Rice's theorem means that you can't really do that without restricting/annotating semantics like Rust does.

No need to imagine, It's becoming reality -> https://internals.rust-lang.org/t/c-lifetime-profile-1-0-a-k...

That is what Microsoft and Google are trying to do with C++ Lifetime Profile.

https://herbsutter.com/2018/09/20/lifetime-profile-v1-0-post...

"Update on C++ Core Guidelines Lifetime Analysis. Gábor Horváth. CoreHard Spring 2019"

https://www.youtube.com/watch?v=EeEjgT4OJ3E

While it might never be Rust like due to language semantics, it is way better than not having anything.

Reply on news.ycombinator.com