How to avoid runtime errors on drones using SPARK (2015)

I think I get what you mean, though. No, you cannot prove "my drone is magical and awesome and will never crash land even if I tell it to, and it takes the best pictures of any drone". You cannot mathematically prove "My on-board sensor will always work and defy the laws of physics and never be inaccurate". You CAN mathematically prove "My drone control software never reaches an illegitimate state, that would cause the system to deadlock or hang due to a software error, making the drone crash in a possibly dangerous, uncontrolled way". You CAN prove "My software will respond within exactly N cycles, at most, to any external incoming sensor signal".

That kind of guarantee is extremely valuable for such systems. It isn't easy (and requires deep, conjoined assumptions and proofs of both the hardware and software), but it's hardly impossible.

And sure, there is only so much the model can prove to you, at some point it has to exist "in the real world". This isn't really a counter-point to the actual formal methods field, though.

To couch it in a more concrete example: when people like Intel say "our CPUs don't catch on fire", they only mean it in a very specific sense. Sure, you can shove newspaper up next to your heatsink, and then when it catches on fire say "See! You were wrong!" But the reality is that their statement is sort of couched in the assumption that, well, you aren't going to do that. It's fairly reasonable to assume some limitations of your model. But this really has very little to do with being able to formalize general mathematics (like complex numbers or finite fields or N-dimensional spaces) using a theorem prover, or whatever.

The people working on formal validation tech are well aware of these limitations in the tech -- the concept of a "Trusted Basis" is a core component of the whole field, around which many techniques revolve.

Systems like seL4 use different approaches, such as "translation validation" which completely removes the need to trust a compiler toolchain, for example (because they do an automated equivalence proof between the generated assembly and a high level specification, they can just not care about the compiler). Interestingly, I know someone who worked on seL4 before its public release -- in fact at several points in the process, they got some parts of the specification wrong. You'd think this would be a death blow. But it still turned out many of these incorrect specifications weren't totally wrong, or useless at all -- in some cases they simply were not strong enough to show other desirable properties. To handwave an example: Maybe they wanted to show "No process can violate the address space of another process or halt it illegitimately" as the specification, but accidentally said "No part of the system can be put an invalid state or hung with core kernel APIs". They aren't the same thing, but it turns out, the second, 'wrong' case still eliminates a useful class of errors (while being a subset of the real specification, because to be unable to interfere with the address space of any other process or hang it somewhat implies you can't hang it, or corrupt it illegitimately, through system APIs.)

It's not (except maybe very rarely) as simple as "proof is bad" or "proof is good". There is a spectrum here along which you can assess their utility -- just like any engineering challenge.

It does take a lot of resources to do though, this is correct. The tech is improving at a pretty fast pace, and we're already getting to the point of having fully verified compilers and OSs (CompCert, seL4). It is certainly becoming more viable to do this all the time. It's still not easy, and takes a very, very different set of skills than most engineers are equipped with.

> I guess (just a guess, maybe misinformed) that proper testing and fuzz testing of code is considerably more beneficial than finding bugs by trying to prove it.

For average software, this is true, but it's mostly a cost question. Frankly you don't need mathematics to make shit, insecure software less insecure, you can absolutely improve them without that.

But they don't really do the same thing anyway; proofs show a mathematical absence (or prove a mathematical fact) of certain classes of errors. Fuzzers can only show the existence of certain errors -- they cannot definitively prove anything other than "Some bugs exist".

There was a paper recently (I can't find it) that explored this idea; essentially the authors injected obvious, "should-be detectable" bugs into many pieces of software, then ran fuzzers over the software to see how many of the injected bugs could be found. The takeaway was, IIRC: fuzzers didn't find anything close to the real number of injected bugs, not even considering bugs that weren't found and were not injected by the authors. Which means that despite their automation triumphs, we could very easily be missing real, catastrophic errors, at a very high rate.

Fuzzers haven't definitively proven that there isn't another Heartbleed out there, somewhere.

CppCon 2016 is full of talks regarding how to write type safe code, how to prevent memory corruption, how to tame UB.

What does the C community? They drop the C99 security guidelines into an optional annex in C11!

Another proof how the C community is deaf to how C's use is hindering how industry.

Meanwhile C++ community is aware of the safety issues it inherited from C and is trying its best to improve safety.

In C++ I have access to:

- Type safe strings via std::string

- Type safe dynamic arrays via std::vector

- Type safe static arrays via std::array

- Type safe enumerations via enum class

- Type safe out parameters via references

- RAII

- Controlled access to unsafe operations via encapsulation and overlooded operators

- Type safe IO via iostreams

- Type safe metaprogramming via templates and inlining

- The preprocessor should be used just for including header files

So no, they aren't the same thing in security mentality.

Modula-2 and Pascal dialects were as fast as C back in the 80's and 90's, before the industry decided to settle on C as UNIX got adopted and just like node.js and JavaScript, C escaped outside UNIX.

The quality of C code in home computers was a joke even to junior Assembly programmers.

Of course C is fast in 2016, with UB all over the place and 40 years of research in compiler optimizations taking advantage of it.

> And how much of CVE-worthy software is not written in C ?

And how much software is written in C without any need of doing so?

But never mind, the only way to convince you guys is to have CVE meltdown after meltdown, until the industry has lost so many millions that it will eventually change.

Millions of dollars and euros are spent in language research, how to make C code safe, since the language got adopted.

Companies sell static analysers, memory corruption tools.

CPU manufactures like Oracle and Intel were forced to add instructions to handle fat pointers.

Research is spent creating processors like the CHERI.

I wonder how much money would have been saved if a sane language would be used instead....