Coverity is really good. It is a pity some of its advances, effective in practice but not really "publishable", will forever remain as proprietary secret.
Source: I worked on static code analysis product and we extensively black-box tested Coverity.
Source: I worked on static code analysis product and we extensively black-box tested Coverity.
As best I can tell, most of the warnings are either things that can be figured out for a single translation unit and (some) compilers will eventually incorporate as warnings, or things that can only be figured out by analyzing / linking across many translation units - which a compiler can't do, but the actual "advance" is simple enough if you have all the function definitions to hand.
In other words, all the advances are in warnings you don't see. Yes, all Coverity warnings you see, are simple. I agree.
Quoting from https://cacm.acm.org/magazines/2010/2/69354-a-few-billion-li...
> Since the analysis that suppresses false positives is invisible (it removes error messages rather than generates them) its sophistication has scaled far beyond what our research system did. On the other hand, the commercial Coverity product, despite its improvements, lags behind the research system in some ways because it had to drop checkers or techniques that demand too much sophistication on the part of the user.
This is correct. Coverity does a bunch of specific analyses designed to eliminate False positives. This includes analysis to determine which data states are not possible for a given code path, and doesn't report those specific issues. This also works using data across function calls.
For C/C++/C#, Coverity has by far the lowest false positive rates, which make it probably the best in class for those languages.
Disclosure: I used to work at Coverity.
I now have quite different perspective wrt false positives; that false positive rate is not important. It's all about perspectives. Name the tool bug search engine instead of bug finding tool. You rarely look beyond the first page of search engine result. Develop a ranking algorithm such that all alarms in the first page is relevant. You can use any probabilistic voodoo to rank. Etc. I don't know whether this can work. But I think it's worth a try.
Of course, this is not obvious to people who haven't tried, so it is reported over and over and over. The most recently, from Google. Lessons from Building Static Analysis Tools at Google (2018). https://cacm.acm.org/magazines/2018/4/226371-lessons-from-bu... I mean, I could have told them.
Select quotes:
> Unlike compile-time checks, analysis results shown during code review are allowed to include up to 10% effective false positives.
IMPORTANT NOTE: 10% effective false positives means much less than 10% false positives! In above quote, Google defines true bugs marked as not-a-bug by a developer as effectively false positives. If what you say is true, but users misunderstand, it doesn't count.