Hacker News Reader

Comment by _bxg1 | Hacker News Reader

I have to wonder if Rust is putting pressure on C/++ to have more static analysis (while at the same time blazing trails in what's possible in that space, and what's possible in terms of error message helpfulness). I think it's a great idea to start baking these things into the compiler, even if it will never be 100% free of false-negatives because of the limitations of what the language can express and guarantee. Still seems like a great way to eliminate a lot of common problems, as a default across the ecosystem instead of as an extra step.

View on HN

It's clang, not rust.

And clang's analyzer has different UI concept via web, which is far superior. And for the screen valgrind has a far superior solution. I don't see the advantage of gcc's analyzer yet. Far too verbose. and the most important errors, like wrong optimizer decisions based on their interpretation of UB code are still silenced.

I'd love to see a compiler warn me that it's doing something potentially unexpected due to UB considerations.

Luckily, compiler authors have gotten the message and are starting to improve on this situation slowly.

clang has UBSan, which adds runtime checks for detecting various kinds of undefined behaviour: https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html

gcc also has ubsan since gcc version 4.9. Many of the sanitizers are shared by the clang & gcc projects. Some functionality, like emitting errors on unsigned overflow, is confined to clang.

That's not helping. I'm talking about wrong decisions, made during compile-time optimizations, like assuming dead code or a value being NULL, and then ripping apart the written code. Without warning.

Or the famous optimized away memset call. Which is a security issue. At least a warning would be in order. Or at least an analyzer warning.

That is intrinsically limited because UB are runtime issues (otherwise they'd be compiler errors) and the compiler simply assumes they don't happen, plus many of the "UB-related" optimisations (in the sense that these optimisations change the behaviour of non-well-formed programs) are surfaced by other optimisations (largely inlining).

Problem is there are a lot of cases where the compiler doesnt know if the undefined behavior would happen in the real world. That is there is undefined behavior if some input parameter is out of range, and no way for the compiler to know that the input will always be in range - something I as a programmer with more global knowledge knows for sure which is why I don't check it.

http://blog.llvm.org/2011/05/what-every-c-programmer-should-...:

> warning: after 3 levels of inlining (potentially across files with Link Time Optimization), some common subexpression elimination, after hoisting this thing out of a loop and proving that these 13 pointers don't alias, we found a case where you're doing something undefined. This could either be because there is a bug in your code, or because you have macros and inlining and the invalid code is dynamically unreachable but we can't prove that it is dead.

Gcc already (gcc-8) reports possible undefined behavior for loops and these are mostly out-of-bounds in my code. I like that a lot, actually, it save a ton of time.

Yes, I tried the ReSharper Inspector tool, and it uses clang to find all sorts of code issues, with a friendly UI.

I think you mean "false-positives". Most static analysis tools are "conservative": will throw more warnings than they can prove. But assuming they're "sound" (and don't "give up" because of time/space constraints), there'd be no "false-negatives".

Vanishingly few (maybe even zero) industrial static analysis tools are sound. FPs are too common, especially for something like C++ where common code patterns can only be soundly handled by just forcing everything to Top. Good luck with your sound interprocedural pointer analysis as soon as somebody makes an indirect call.

I think you mean industrial static analysis tools aren't complete. Sound means no FNs, Complete means no FPs.

No I meant what I said. Virtually no tool is sound (zero false negatives, lots of false positives) because in practice false positives just make people stop using tools when they are too common.

At my previous job, we used an internally developed static analysis tool. While false positives were a problem, the team responsible for the tool took bug reports and fixed these issues. There was a way to silence false positives as well as a mode to only see new (potential) problems. Once code was clean, the false positive rate was low enough to not really be a problem.

The bar of zero false negatives is not useful for me when the alternative is to rely solely human code analysis and tests.

One thing I learned from using this tool is that static analyzers need to be able to configured to know about side effects of functions that are in libraries. For example, if a library function will free memory, it is important that there's a way to tell the analyzer about this behavior to avoid false positives about leaks.

Proper design, careful coding, static analysis, code review, and various forms of testing all have roles to play in delivering reliable code.

But you can't fix all false positives with bug reports. Its fundamental if you want soundness. And for lots of properties (like I said, pointer analysis in C++) the only scalable and sound techniques basically havoc as soon as interesting things start to happen. The community even has a term (soundiness) to describe how everybody throws out certain kinds of soundness (indirect calls through function pointers, reflection in java, tons of features in javascript, etc).

Requiring the user to add piles of annotations still sucks, even if you are smart and do something like abductive inference to identify minimally required annotations. Even something as simple as the checker framework in Java has trouble getting people to write correct annotations and has a few corner cases (what is the nullness property of a field in code reachable from a constructor, for example). Nobody does this stuff correctly for something like dataflow annotations on methods.

I've got a PhD in this and develop a static analysis tool for my job. I've never seen a productionized, fully sound tool for general purpose static analysis of general programs that solves problems more difficult that slightly augmented type checking.

So if industrial static analysis tools aren't sound, which means they produce false negatives (ie. they can pass programs with logical errors they're meant to detect), how do you use them to assure code quality?

Same way you use anything. Testing isn't perfect, yet it is still useful. Fuzzing isn't perfect, yet it is still useful. Unsound static analysis techniques aren't perfect, yet they are still useful.

The alternative is that your abstract interpretation is pinning to Top very quickly and then throwing false positives everywhere.

You can't assure code quality, you can only use them to make you code is a bit better because you fixed issues found by the tool and asymptotically approach ideal code and lower the probability there are serious errors. AKA better than nothing.

Which is why some OS vendors just gave up and are going with hardware pointer validation nowadays.

If all goes according to plan, I guess in a couple of decades there won't be any modern computer not making use of it.

Internet and ramping project costs due to CVE fixes are what is putting pressure into adopting static analysis and hardware memory tagging, not Rust.

https://msrc-blog.microsoft.com/2019/07/16/a-proactive-appro...

https://security.googleblog.com/2019/08/adopting-arm-memory-...

Lint exists since 1979.

Hardware memory tagging is not a panacea, it just catches egregious memory errors.

Indeed, yet it does a better job than the mythical developer that never gets pointer arithmetic wrong.

Rust’ll outdo both.

As much as I would like to nuke C and any derivative that has copy-paste compatibility with it (C++, Objective-C,...), no one is going to rewrite Linux, BSD, OpenGL, Vulkan, DirectX, Windows,macOS, iOS, Android,.... in Rust.

So I do welcome hardware memory tagging, even if isn't perfect.

Apparently Google thinks the same way, given the Android is getting hardware memory tagging as requirement on ARM, with C and C++ still having more visibility than Rust on Fuchsia.

Just like Microsoft is doing by pursuing research in Project Verona and CHERI CPU, Checked C (hiring now for the team), Visual C++ Core Guidelines static analyser, focus on .NET, instead of going full speed with Rust.

Similar story can be told about Apple's ongoing changes.

Reply on news.ycombinator.com