"One of my fundamental rules of system design is when people keep doing it wrong, the people are right and your system or idea is wrong."
I am not sure under what circumstances this is true, but for security it is definitely false. People keep forgetting to sanitize inputs, for example. Does that render input sanitization a broken idea?
If everyone forgets to sanitize input, that means our current processes are fucking broken.
I like to imagine if this excuse happened in aviation how it would sound.
“Everyone forgets to put down landing gear before landing, so does that mean landing gear is a bad idea?”
No, but it does mean the process for landing is catastrophically broken, and needs to be overhauled. If that happened twice, we’d implement strict rules around landing like checklists with standardized procedures to have a second person verifying that the process was followed and each task was accomplished, and defining a “sterile phase of flight” where talking about anything other than the landing process was disallowed.
So, input sanitization isn’t a broken idea, but the fact that it keeps going wrong clearly demonstrates that out process and culture around security are catastrophically broken.
I absolutely agree that psychological acceptability is key for building secure systems, and that the human factor must not be neglected. I also agree that process and culture around security are deeply broken. I just do not agree with the idea that if all are doing it, there must be some deep truth beneath. Yes, people habits must not be ignored, but to claim that an idea that goes against those habits is wrong, as the author claims, is inaccurate.
I think it does say that there's a deep truth underneath. That deep truth is that the "correct" way is not good enough.
The way that it's not good enough may very well be that it doesn't mesh well with how people work or think. That aspect is as important as any other aspect of design.
Hypothetically, lets say you design a system that, for every legitimate 1000 users accessing it, 999 fail to understand it enough to gain their legitimate access.
Would you still be comfortable telling the people that paid for the system that it's not the system that's wrong, but the users?
I'm for input sanitisation, I'm against having it optional and left up the the developer.
Typing on a phone makes me unnecessarily brief, apologies.
Well ... yes? If people are forgetting to do something that is required, then that something is explicitly needed.
Better to make that "something" implicitly added in the process no matter what the user does[1]. Or make the process break if the user "forgets"[2].
All difficult things to be sure, but easier than expecting the user to remember which of the 100 different NON-DEFAULT sanitation packages to install, configure and use, for output to HTML, SQL, JSON, Logs, and more.
If the language makes it easy to do, and there is very little blowback in terms of security[3], then users are gonna do it.
[1] Auto-sanitise strings, obviously. I dunno how you'd actually do this though.
[2] When running as a web-service, have Django/whatever-framework configure Python/whatever-language to emit warnings whenever string interpolation is used without any escape function.
[3] For HTML, the exploitation of string interpolation are few and far between; they're so rare as to be lost in the noise. Hence, users don't use it. For SQL, injection was a real problem, with the risk of getting pwned on string interpolation being close to 100% on a good day (and actually 100% on a bad day), and so users actually used the mitigations there were.
What I want is to have my runtime/framework/language drop me into the pit of success (a popular phrase, might even have it's own Wikipedia page)
Maybe it's not possible to do any better (no magic language that can tell what a given string of bytes will eventually be used for) but that doesn't change the fact that the reason inputs don't get sanitized is because the system requires an unreliable component to be reliable.
It's also a bad example because it's not really a choice (no magic language).
It's a different way to address the problem of the unreliable component than just requiring it to be magically reliable, or replacing it with something else that's reliable (automation, ie magic compiler or runtime), by adding procedure and redundancy and cross verification so that the final output is reliable even though all the individual worker parts are not. It's just ECC in human scope.
So I guess you're right that it is an example of the article's assertion that "If people always do something wrong, that is evidense that there is a system in place which results in people doing that thing wrong, and that system is wrong."
The system is the overall norms of software development, and the wrongness is that it doesn't include something equivalent to a pilot checklist culture around the important bits, even though everyone knows that those bits are important, and knows that humans are unable to be reliable.
Definitely sucks that overall software development norms do not take more care to protect the important bits. Hard for the market to reward secure designs, because the effects are latent, because the competition is not any better and everyone claims their product is secure. If a market required data showing security and if the security industry came up with cheap effective solutions, things might be different.
I do not view the system that birthed the checklist concept as being flawed or wrong, though. I agree that if people always do something "wrong", then it requires close scrutiny. It is just that different systems have different requirements and levels of redundancy. The checklist came about when people realized that humans make costly mistakes without one. Same deal for input sanitization (which many on this thread are not liking). OWASP is of the opinion that input sanitization is worth mentioning, but they would probably agree the whole thing is broken...
That's why there's also the buddy system, procedure aka habit aka ritual, and culture aka the pressure of norms.
No single thing guarantees much, they are all just pressures that some people are immune to, or some can't always be applied (buddy system requires buddies), but they do all have their statistical effect, and the more of those you pile on, the less likely it is for something to make it past all of them.
Say someone is a total lone wolf, not part of any teams or communities, utterly immune to shame from not conforming to everyone else's expectations, the system still works because everyone else shuns their work because it came from such an unhygenic source, or, they take on the job of doing what the author didn't, because one way or another they simply can't be seen using dirty software in their own work. They either don't use it or they launder it themselves.
At least, I assume it's diffucult for a total cowboy scofflaw unsafe pilot to get a job doing any piloting that matters anywhere. I bet even the military demands people be in control of their intentiinal crazy unsafe flying.
Not just because of any rules but the entire culture made of the entire rest of the population that you can't buck individually.
And we do actually have a little bit of this in some, maybe most companies, where developers generally don't push code directly to production but must pass through at least one other reviewer to merge. But the company as a whole gets to do whatever they want in private, and they don't do that same sort of auditing on all the stuff they use that came from elsewhere, and thst's only big companies which are still probably a minority of all developers and projects. Even if I'm part of such a culture at work, I'm still not the rest of the time unless I choose to be.
Could a compiler or static analyser not work backwards from execv() params?
I guess it's not as simple as that because really all functions matter not just the exec/system ones. Anything that merely reads or writes memory at all, could be used eventually to disclose a secret or alter behavior. Without ever doing something as obvious as executing a shell command, I could maybe change the address to a function, causing the function to behave differently, skip some bit of important logic that sometime later in some totally unrelated process allows me permission to access a file or interface I shouldn't etc. In the end, I guess there is no function anywhere that doesn't matter, and so the backtraking process would just end up including everything. Not useful.
You would need a "mark of the web" bit of metadata attached to every single variable and block of disk. That would need to be baked in to the hardware platform as an extra bit. Everything on the motherboard, ram, cpu, storage everything is 9 bits instead of 8, with 8 payload and the 9th is the tainted mark. Not just every byte of memory, but every part of everything. The disk bus has the extra bit for the data, all the disk access instructions include the extra bit to taint the issuance of the instruction itself, which also taints the data the instruction handled, every cpu register, every address, the address itself as well as the data at the address, etc...
These could all be 65 bits for 64 instead of 9 for 8, but only if the platform was made incapable of operating on anything less than 64/65 bits. 64/65 would have to be the fundamental "char" for the platform.
Even then, I bet it would all have to reslove out to every byte or word ends up having that bit set for one reason or another. Ultimately 100% of the data comes from outside and comes from people, or was somehow derived from data that did.
Though maybe we could simply artificially bless certain data even though it is software or firmware or data that humans wrote and loaded in, and use that to keep track of the rest in a more useful way.
Rust in hardware?
Somewhat different plan, more like, ACL-in-hardware but similar idea having hardware that tracks extra metadata bits every step of the way and at the lowest levels.
The developers aren't all idiots, they are following your incomplete documentation and using the buggy and incomplete example code you provided. Or they are using the buggy example code they found online because you didn't provide any. If your API isn't useful without first taking a college course on the problem domain then that is a problem with your API and probably documentation. The whole point of an API is to encapsulate the complexity and provide the user a tool they can use without having to first learn everything about it.
Possibly. You could prevent careless use of unsanitized inputs with a type system. Fail-safe designs should be more common.
The input sanitization system could be improved by requiring sanitized inputs, reminding people that they need to sanitize inputs, automatically sanitizing inputs, etc. Clearly the current system is leading to a lot of insecurity.