Comment by nickpsecurity | Hacker News Reader

Comment by nickpsecurity | Hacker News Reader

Parent Full thread

The military and defense contractors did empirical studies of various programming languages and their defect rates to put that to the test. That was mainly in 80's and 90's. They compared C, C++, Ada, and Fortran mainly. C usually had double the defects of the rest and with more severity despite pro's writing it. Ada usually outdid all of them with one showing Ada and C++ developers having same defect rate.

So, when we use evidence instead of feelings, what you said is a myth that's been debunked repeatedly in many ways for decades. And, yet, people repeat it. No, C use usually results in problems that safer, systems languages before and after it had less or none of. Despite professionals using it. The solid code is always an outlier.

Most amusing thing is Thompson and Ritchie, with help from Pike, later designed what they thought was perfect language with perfect set of features. The result, Go, was basically like Algol subsets that preceded C mixed with Pascal developed around same time. But, C compiled and ran on a PDP-11 fast. So, it's the best. ;)

Yes I remember the studies (and have done my fair share of defence s/w).

And I am not claiming that C is the best tool for the job because of the language.... its because of the 'environmental' factors, human resources, community, momentum, codebases, portability, etc.

I whole-heartedly agree that there are better languages out there, but the fact remains that the flaws in C are easily managed.

It would also be interesting to produce a study of the use of C specifically in embedded systems where the style of code is very much different than large desktop systems (i.e. much less heap usage and dynamic memory allocation for example).

"And I am not claiming that C is the best tool for the job because of the language.... its because of the 'environmental' factors, human resources, community, momentum, codebases, portability, etc."

I agreed with you on that in another comment. Yep. It's also why NASA often uses it. The tooling covers dark corners pretty well these days.

"It would also be interesting to produce a study of the use of C specifically in embedded systems where the style of code is very much different than large desktop systems (i.e. much less heap usage and dynamic memory allocation for example)."

I'd love to replicate the old studies on modern languages with modern tooling for both systems programming and the embedded subsets like MISRA. We'd definitely learn some stuff from that. Further, I'd like to see specific metrics like in Ada/SPARK/CbyC example below on where defects were introduced or corrected with each technique and phase of lifecycle. Would tell us how low-level features, subset rules, and tooling interact with accurate assessment of how much problems they pose for real instead of in theory.

http://www.sis.pitt.edu/jjoshi/Devsec/CorrectnessByConstruct...

This is the thing...

Modern embedded C programming has evolved significantly in the right direction since studies were done and so I would love to see someone spend the time looking at 'modern' embedded C codebases.

Static analysis is now widespread and not theoretical, coding standards now take security seriously (tho can be outdated). Drawbacks to certain techniques have been surfaced and recognised. Tooling is much better, portability is recognised as a good-thing, pointer usage is now minimised when possible and contained to areas where they are appropriate. Casting is frowned upon, macro-magic is frowned upon. 'Clever' code is frowned upon.

Basically, I would think that certainly in the embedded domain, standards are now such that the story would now be significantly different.

Any recommendations on static C analysis for embedded systems?

No one-tool does the whole job. Setup your CI system to use multiple tools (Lint, CLang-analyser, cppcheck, etc).

Dont trust just one.

Pretty much what we already use.

I was hoping for something new.

Here you go. Experiment away and give them constructive feedback as they need it. Esp Softbound and SAFECODE.

https://news.ycombinator.com/item?id=11770726

Thanks!

I'll look into those.

Have you tried Astree Analyzer? Papers I read indicated it was one of best but hard to find industry people to confirm or reject it. Want someone to clone it for FOSS. Meanwhile, I found these free ones for you two that have each found bugs in code with focus on minimal annotations. All are academic prototypes but Saturn was used on Linux kernel.

Saturn http://saturn.stanford.edu/pages/overviewindex.html

CSolve http://goto.ucsd.edu/csolve/

Calysto http://www.domagoj-babic.com/index.php/ResearchProjects/Caly...

Most academics are instead making compilers that translate C code into something mostly or totally safe while pushing the performance hit downward. Here's two of the top for you to try on various codes (or improve):

https://www.cs.rutgers.edu/~santosh.nagarakatte/softbound/

http://sva.cs.illinois.edu/downloads.html

Reply on news.ycombinator.com