Would you not want to try out what happens if I pulled a door you built that is only meant for pushing? There will always be that one person who ignores the sign and tries pulling just like there is always that person who uses your function in an unexpected way.
If a test suite covers all aspects of an implementation, it's basically a second implementation, and then the question becomes whether the time spent writing this test code would actually have been better spent looking over the actual code that defines your algorithm. If we can't detect errors in code by reading it, I don't think tests will help us much.
Furthermore: if our algorithms require a second "test implementation", shouldn't we also write a test for the test, to confirm that the written test actually tests correctly? What good is a broken test? If you've built a plywood door, and a machine for testing that door, wouldn't it be useful to have a machine that tests that your testing machine actually tests the door properly?
All jokes aside, I really enjoy automated, high-level tests, which mimic actual user behaviour. I think this is different because we're no longer implementing an algorithm a second time around (as a test), but rather encoding user behaviour (of the algorithm) into an algorithm (new code).
The OpenBSD code has been audited many times by very experienced and careful C experts. Fuzzing apparently found the very few things they hadn't already found by reading the code.
Their formula for what is worth essentially boils down to "if this approach can find at least one bug no other approach would do it, we will use it".
That's definitely comforting, but I guess my point is that the fact that bugs can even hide in plain sight in the first place is the root cause of the problem. If we can look at a specification of what something is supposed to do - the code - and not see that it does something entirely different, then the problem is with the programming/specification language.
The very purpose of a programming language is to make the specification of computer programs readable (rather than reading assembly). If a specification language needs a a computer program to go through the specification and see if it specifies things that we didn't intend to, then that specification language misses the very point of being a language in which to specify things in the first place.
If a specification language requires the writer/reader to be aware of the entire specification in order to verify whether an isolated piece of specification is valid, that specification language is not of much value, I would argue.
It occurs to me that the problem with traditional languages is that they do not allow the direct reference to information (values); only to variables which contain values (information). In Haskell, if you want a place to store information you create a TVar (variable), while you are forced to do this, always, in traditional languages, if you want to work with information/values properly. You're forced to keep information in a register if you want to manipulate it, and always refer to the register, when the information it contains is what you're really concerned with. Having to both think about the values/information and where you stored this information is double work. Why not reference the information directly?
Syntax is not really an issue for any programmer with a reasonable amount of experience. I very rarely have trouble accurately transcribing ideas to code. More often, my ideas are faulty.
Physics engines in games are a perfect example. They're a perfect use case for functional programming, there's not a lot of complex state ideally, and yet even simple simulations are notoriously fraught with unexpected behavior. See: thousands of YouTube videos of physics glitches in games.
> Syntax is not really an issue for any programmer with a reasonable amount of experience.
> They're a perfect use case for functional programming, there's not a lot of complex state ideally,
> and yet even simple simulations are notoriously fraught with unexpected behavior. See: thousands
> of YouTube videos of physics glitches in games.
If an implementations of 3D scene rasterization contains glitches, the language used is most definitely the issue, because producing glitch-free 3D animations is a solved problem. No programmer writes a physics engine to produce glitches, so if they magically appear, even though they aren't visible in the spec, the spec language is faulty.
I guess I'm arguing that the problem is that most popular languages allow you to write programs/specs that are invalid (will crash when executed). So we're writing specifications in languages where we can't even say whether the specification is valid or not (actually implementable). If humans produce buggy programs, that is proof that we're using the wrong tool, I'm arguing, because no one intends to produce bugs.
Haskell is a bit scary, at least it was for me in the beginning, because it can seem so hard just to get your program to compile. But that's because the compiler actually verifies that your program is valid - that no unhandled exception will occur. If people are having writing valid, purely functional programs, perhaps this is, in part, because their idea is unimplementable, and Haskell is telling them this via not being able to compile your spec, while other languages inform you of this via an unhandled exception that crops up a year later, after the code is in production.
That's not actually correct. The compiler verifies (modulo your transitive use of unsafe primitives) that certain things won't occur. Unhandled exceptions are not one of those things.
You're right of course: no compiler can guarantee lack of exceptions when it comes to IO, but I would argue that GHC, indeed, does verify that - unless a function explicitly throws an exception - no exception will occur. This is very different from C, where an exception can be the result of just about any operation (if you do something wrong), rather than only an effect that appears when you use "throwIO" (or similar). It's the difference between exceptions being used as a tool by the user (programmer), rather than a way for the compiler to tell the user (after your program is compiled and is running) that it has no idea what to do now, and will abort. In Haskell I write the "error"/"throwIO" statement causing an abort, in a C program I might have intended something completely different, but the compiler throws one at runtime.
We mustn't mix up IO with everything else, because reality is inherently unreliable, and it's just the nature of it that a certain file won't necessarily be readable tomorrow - we can never know that. But we can know whether a pure function, sans compiler bugs, will execute without throwing an exception. And I would argue that adding "error" or "fail" does not constitute an unhandled exception, as that statement was put there intentionally by the author of the program (perhaps to test out something).
I'm saying that popular physics engines generally have bulletproof, tried-and-tested code, and are even implemented in a very functional way on a conceptual level. Despite this, they exhibit glitches. The glitches don't appear magically out of the spec language, they are an inevitable result of the discrete nature of realtime physics simulations.
This is meant to be a counter-example to your assertion that bugs come from miscommunication between computers and humans. My argument is that more often than not, we communicate our ideas perfectly, but our ideas are flawed. In the case of physics engines, they are flawed by design in order to compromise accuracy for performance.
> In the case of physics engines, they are flawed by
> design in order to compromise accuracy for performance.
> The glitches don't appear magically out of the spec language,
> they are an inevitable result of the discrete nature of
> realtime physics simulations.
And then you run the tests and they will, hopefully, tell you if the how matches the what, and if it doesn't where the two differ, i.e. where the the code doesn't do what you intended...
If you aren't in the habit of using debris for structural items, sure, spend some time testing failure modes. :)