The Insecurity Industry
edwardsnowden.substack.com
edwardsnowden.substack.com
Where there is no liability, there is no accountability... and this brings us to the State. "
Yep, this definitly needs to eventually happen.
Sure if you add more complexity, you add more attack vectors, but there's an easy way to reduce your legal culpability there: just don't collect any PII. Even in the scenario you propose where anonymous HTTPD logs are a liability (which... yeah, is not going to happen any time soon) the solution is simple: turn off logging. If the legal precedent is established, the defaults of our software will change to match.
You're coming at this whole thing from the perspective that wise and sane rules would be put in place and then sanely enforced for the welfare of everybody—by the same US government that told people not to wear face masks to protect against covid, while also shipping defective covid tests from the CDC and prohibiting the use of any other covid tests. And that's a case where nobody was in a position to profit by making the rules hard to comply with.
Listen, you know and I know that you can serve a personal website perfectly well with /var/www and a stock Apache config. But the proposal we're discussing here is precisely to take that judgment call away from people like you and me and give it to people like Donald Trump, using laws written by, most likely, lobbyists from Oracle and Microsoft.
> Listen, you know and I know that you can serve a personal website perfectly well with /var/www and a stock Apache config. But the proposal we're discussing here is precisely to take that judgment call away from people like you and me and give it to people like Donald Trump.
Let's revisit that proposal:
> 1. defining legal liability for bad code in a commercial product
> 2. making [website operators] legally liable for any and all leaks of our personal records that a jury can be persuaded were unnecessarily collected
I believe we both agree that neither of these technically apply to the personal website scenario (static hosting not collecting any PII).
So your argument as I understand it is: in order to make the above liabilities legally enforcable for scenarios where they do make sense, we will end up with regulations similar to those for handling "sensitive" data (such as financial/medical information) being imposed on _all_ software / online services (such as basic static websites). This will happen because laws will be written in an environment of near-total regulatory capture.
This argument is plausible, but it relies on a bit of a non-sequitur: expanding the scope of data collection/handling regulations will inevitably extend to regulating the publishing of software.
It might be in the interest of the current software behemoths to push for such a system, but I don't really see it. They derive too much economic value from the current "free as in lunch" open-source to shoot themselves in the feet like that. It seems more likely they would:
1. try and narrow the scope of their own liability (by heavily constraining which categories of software carry that burden) 2. try to minimize the costs to themselves (by demanding compensation from governments for the work required to meet those regulations) 3. try to offload liability to vendors (who can then demand compensation for taking on that liability).
Points 2 and 3 could be a large cash cow for free and open source software, though I doubt many will be able to successfully capitalize on it.
It's true that imposing liability for publishing defective software is logically independent from imposing liability for collecting unnecessary PII that leaks. But pjmlp's quote from the article we were commenting on explicitly proposed doing both of these:
> For example, if you want to see Microsoft have a heart attack, talk about the idea of defining legal liability for bad code in a commercial product. If you want to give Facebook nightmares, talk about the idea of making it legally liable for any and all leaks of our personal records that a jury can be persuaded were unnecessarily collected.
So my argument does not, as you say, "rely on a bit of a non-sequitur: [that] expanding the scope of data collection/handling regulations will inevitably extend to regulating the publishing of software." The proposal in question is to both regulate software publishing and also regulate data handling, so it's irrelevant whether or not the scope would thus "inevitably extend" from one to the other.
Probably it is true that the most favorable situation for the current incumbents would be to have no liability, as at present, or as minimal liability as they can get away with. But the second-most-favorable situation, and one that is definitely politically viable even if the current situation is not, would be to have a regulatory regime that raises the barriers to entry for new entrants as much as possible and prevents disruption to their markets, by enshrining in law the particular way they're doing business today: AI melody recognition for prior restraint of free speech, combined with armies of outsourced moderators to watch for terrorism and pornography, centrally-controlled app-store platforms, locked-down end-user hardware (with a grandfathered carve-out for desktops and laptops), real-name policies, fax-us-your-passport ID verification, "two-factor" authentication that turns out to be one-factor, and so on. Anything that encourages you to post stuff on your own blog or website would be a big drawback for GitHub, YouTube, and Fecebutt.
Are you going to put up with accountability that you might have misconfigured something and it allowed attackers to scam people or serve porn?
You are perfectly sure that you are going to keep your small site updated all the time and you won't forget about it?
Because that is where it is going - it is not just code that can be vulnerable - but also combination of different software, combination of configurations. If you install 2 applications they might interact in a way that makes your system vulnerable.
Software is infinitely complex we can cut down complexity but then anything that is useful and complex will cost a lot more.
What if somebody steals my kitchen knife and uses it as a murder weapon?
> Are you going to put up with accountability that you might have misconfigured something and it allowed attackers to scam people or serve porn?
Yes. This is (and always has been) the price of operating a website on adversarial public networks. We established relatively simple ways to make this possible even for individuals decades ago.
> You are perfectly sure that you are going to keep your small site updated all the time and you won't forget about it?
As I replied to the sibling comment, when was the last time there was RCE for Apache or Nginx configured to serve static files from a webroot? We are talking about personal websites here after all.
> Software is infinitely complex we can cut down complexity but then anything that is useful and complex will cost a lot more.
I think I disagree with you on where the threshold of usefulness is.
https://www.vice.com/en/article/qj8xz3/a-defunct-video-hosti...
People like to embed things in their personal websites - integrate them with third parties. That is my point.
Now you are responsible for what you embed on your personal website and what you publish.
The guy selling food on the street has the same liability as a restaurant.
Maybe you could make the case that he was following best industry practices in doing so; after all, using Valgrind is a best practice, right? But you could also pretty plausibly convince a jury that he was negligent. Especially if you're IBM's senior counsel. Or, say, RSA's.
Now, is Kurt Roeckx or RSA going to be advising the US legislators who draft this candidate legislation, establishing the standards that they both must uphold?
If someone gets run down by a bicycle that a hobby repair shop failed to fix, it doesn't matter it was done for free by a guy that learned to repair bicycles during late nights.
But yes, lawmakers will decide, and given that they for instance try to de facto prohibit aftermarket OpenWRT installs, I have a guess how they would decide.
The company selling me a car is responsible to validate the security of each piece they got from a third party, and a restaurant is responsible to take care for the quality of the food it buys from the local bazar.
When the wise must obey the commands of the foolish, disaster ensues.
The law makers cannot make the internet safer by one bit. Technical experts can and lawyers would dream to have leverage against them. They should be denied.
It is about minimizing risk and it is a process that acknowledges that risk cannot be removed completely. It just forces you to work carefully and eliminates neglectful practices.
No serious developer will ever commit to ship software free of bugs. On the contrary, that would give people false security, which can in turn lead to further neglect.
Simon Tatham doesn't have any profits, but his PuTTY is installed on every developer's Windows machine.
someone taking non-commercial products and commercializing it is where the line is drawn right?
Instead you can look at existing heavily regulated software markets to see what would happen: medical-device software, avionics software, car engine control units, cryptography before 01996, tax preparation software, PCI compliance measures. A vast wasteland of incompetence, waste, government graft, monopolies and duopolies, truly staggering profits, and easily avoidable deaths.
Consider: why aren't you wearing a Holter monitor? How about an automated electric defibrillator? Why isn't cryptographic security integrated into all the internet protocols?
How's that Bitcoin rollout going in El Salvador?
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Not if the rules are carefully targeted at SaaS and not at codebases. If the rules are targeted at SaaS, the liability is actually lower for open source because of the inherent transparency of everything open source code does.
If I run over someone when driving a car, I'm responsible - not the car maker. If my customer's data is stolen, I'm responsible. Whether the makers of my software are responsible is a contractual matter - and nearly all open source licenses include a disclaimer of warranty and a limitation of liability, including the GPLv3 (see sections 15–17).
That was true in the US until MacPherson v. Buick in 01916 and in the UK until Donoghue v. Stevenson in 01932. Nearly all proprietary software licenses include the same disclaimer, but they are on slightly firmer ground in doing so, since typically those licenses are in fact contracts under common law, while the GPL explicitly purports not to be a contract and is very likely correct about that.
The kind of statutory imposition of liability we're discussing here would have to specifically outlaw such contract terms in order to work at all. You could imagine a statute that would specifically exempt open-source software licenses from that, but as explained comprehensively in this thread, such a statute would certainly not be the one that was passed.
ETA: the first couple of Google results say that no, product liability can't be disclaimed away - particularly when there is no contract or opportunity for bargaining. I am very much not a lawyer but this sounds correct to me (i.e. this is what the law is).
https://www.findlaw.com/injury/product-liability/are-product...
https://www.eltonlaw.com/does-a-disclaimer-mean-you-cannot-f...
> Though manufacturers cannot so easily escape liability, sellers can escape liability by informing the customer before the purchase that a product must be taken "as-is,” which means how the product was found when it was purchased in-store. “As-is” works because the buyer has an opportunity to inspect the product and decide whether to buy it given its condition.
On that analogy, Github and RedHat aren't liable, but the original author of the software still is.
There's also no liability associated with running a website. Simply refrain from collecting data of any kind and there should be no reason to worry.
There are certainly people who would like to make it so that the same thing happens with software and online publishing: a few companies controlling almost all of the activity, and if you release any software or host a blog without working for one of those companies, you get arrested within a month. Other people don't intend that, but advocate policies which would have that effect.
But it weakens your argument a bit. Basically, it would only convince people who are already convinced.
Perhaps try looking for natural experiments, eg compare between countries, or between different sectors.
(Sometimes there's also silly legislation you can exploit for statistics, like the Onion Futures Act (https://en.wikipedia.org/wiki/Onion_Futures_Act) which can help to see the impact of futures trading on commodities.
Perhaps there's some corner of the pharmaceutical market that wasn't hit or was less hit by the Pure Food and Drug Act?)
This is not at all comparable to corporations slurping up all data they can get their hands on for marketing purposes. Modern medicine provides enormous benefit for society. Surveillance capitalism... doesn't. Certainly not enough to justify the massive abuses being perpetrated.
Widespread data collection on the other hand is totally unnecessary and should absolutely be a massive liability for any company that does it.
If you want software where the vendors are liable, you can get that today.
https://madeintandem.com/blog/massive-hertz-accenture-lawsui...
Only contract with companies that either have a reputation for upholding their end of the bargain without a court threatening them (ie most good companies), or only contract with companies where you have a reasonable expectation of being able to win a fair court case.
As an example of a more generalised version of the former: Amazon is pretty generous in their customer service, and you don't typically have to sue them to get them to eg give you a refund.
Reputation is a powerful asset, and companies often want to protect theirs.
(Not always, though. Amazon is less nice to sellers or employees, I think, for example.)
What about those not even doing business with Equifax and getting their info stolen?
I don't know anything about Equifax? What are they doing?
I assume whatever Equifax is accused of doing is already illegal by current laws? Would making it 'more illegal' help?
IIRC it could be interpeted as coving a range of types of harm to people and the environment that is not restricted to control systems.
That's kinda sorta one of the goals of the GDPR. And they go farther as they're liable for any leak of personal data, not just the unnecessary personal data.
d-e-f-i-n-i-t-e-l-y.com
That we have security flaws is always inevitable. Better languages might help but are no panacea.
I agree with Snowden on a lot, but this doesn't solve anything.
The result would be software certificates. By whom? Take a guess.
Nobody can guarantee absolute safety. This is a trap you don't want to fall into.
It would end open source and any independent development. Quite surprisingly short sighted by Snowden.
The problem with iMessage wouldn't be solved by liability. It is a security flaw that cannot be removed by law.
edit: To clarify: I agree with him in the Facebook example. They collected the data for their business and should be liable. "Bad" or "insecure" code is a different matter however.
If they had financial incentives to not get hacked, it would make more financial sense to port non-memory safe c and c++ code to swift and rust. (Currently way too much effort to be worth it.) It would also incentivize better security layers like sand boxing.
On the topic of unsafe language though, he's absolutely right. We don't have to put up with this. We could pass a law and ban new code in unsafe languages from national security threatening devices like phones or cyberphysical devices like self-driving cars and we would be the better for it in under a decade. We could even have taxes to give breathing room during a transitionary period to encourage it before outright banning it, but we don't. We don't because so often it is less of a hassle for the government to trust the private sector that it is to take some real position on the regulatory issues that matter. This will probably always be the case until the government is able to evaluate talent and pay salaries in accordance with that talent as the private sector is.
Yes on theme, but no on ”there aught to be a law” that bans C/C++ because of an evolving goal of memory safety.
When Rust++ comes out surely there will be people complaining that Rust isn’t safe, and so on.
Best case is you make the consequence punishable (as was described in the article).
I know this kind of thing is a lightening rod for a lot of software folks, but it does have the nice property that it's proactive; you don't have to wait around for the obviously-unsafe practice to blow up in someone's face before you can do something about it.
And no one wants to have to get licensed to write some crud app in NodeJS. But maybe these kinds of high-stakes modules related to encryption, security, and so on could be a place to start with it; if you're going to work in these areas, and offer your work to the public (for money or otherwise), then you either need to be licensed by the professional body, or clearly advertise in the header of each file that the work has not be overseen by a licensed professional.
That's the kind of arrangement that could do things like specify languages, methods, interfaces, and so on.
Don't worry, the pricing will be FRAND (fair, reasonable, and non-discriminatory). You just won't be able to see the compiler and runtime sources or follow stack traces into stdlib code. For security reasons. Defense in depth, y'all.
For example, here's the Professional Engineers Act in Ontario: https://www.ontario.ca/laws/regulation/900941
None of it has anything to do with which tools or methods are used for the practice of engineering— it's all defining jurisdiction and constitutional meta-details about how the leadership is to be selected, term lengths, etc.
So no one can achieve regulatory capture about what kind of concrete is to be used in bridges in Ontario by lobbying the Ontario government; you'd have to lobby the PEO. Maybe your argument is that they're effectively equivalent, but they're really not— the PEO is a pretty different kind of organization from just another government office.
Yes, because this has stopped them doing dumb things before. The gov't will still be in charge of selecting the people in the regulator. The lobbiest will still have influence. There's just no way around it.
Are you really doubting that this is something that "might" happen?
Edit: FAA allowing Boeing to self-certify 737MAX. FDA allowing/not allowing trials of drugs, or allowing a drug meant for one thing to be tried for something else totall untested (ex: AZT).
I don't know that bodies like FAA and ERCOT are really comparable to professional associations; the market conditions make them especially vulnerable to corruption because they both "oversee" such a small number of large players, so you end up with a revolving door. In any case, these are also odd examples to bring up, because in both cases their failure was not about market capture, it was about failure to protect the public. So it seems your argument is amounting to "the regulation provided by the FAA isn't perfect, so it shouldn't exist, just like regulation for security-critical software shouldn't exist."
I specifically asked for examples from "law, medicine, trades, engineering", because those are cases that I feel are much more aligned to what it would be with a professional body overseeing practices for secure software development— they're cases where you have a large number of mostly small-time practitioners, and where the professional oversight mechanism is working in terms of enforcing safe and consistent practices, while also evolving those over time in response to changing conditions.
This is far from what I'm suggesting. I'm just saying that if it is a gov't regulated anything, those regulations will incur wacky decision making due to the influence of outside money. Nobody likes to be regulated against, and if they are in the position to do so, they will use any mechanism available to them to keep the status quo.
I'm also suggesting that any gov't regulation body is not always the panacea people may be dreaming it will be. Anytime a regulation body is proposed, I don't have rose colored glasses. I'd rather be pleasantly surprised that something turns out to be a good thing than having high expectations crushed.
I have less experience with other trades, but you did call out construction separately. My family comes from construction backgrounds at various levels. The 80s in the US saw a boom in the 20 story building construction, and then saw a total collapse (no pun intended) in the construction industry. There are lots and lots of building contracts won by lowest bidder, and the only way to do that is cutting corners somewhere. Usually in quality of material, or reducing the "over-engineered" portions to the point of risking saftey, etc.
It's also widely known in construction that the permitting offices can be gamed. Talk to the right people with the write phrases. NYC is infamous in that people playing by the rules get absolutely nowhere. You have to start spending cash and using influence to get things done. It's all just pointless to being corrupt.
Programmers aren't that good at it either.
Yes C/C++ have more footguns than Java but there's no "hard line" in the safety differences and there are real and important things that need doing that it's not always clear can be reasonably done in another language.
If you haven't, I'd encourage you to read the paper "Some Were Meant For C"[0] on why C still doesn't have a real replacement (though it could in the future).
[0] https://www.cl.cam.ac.uk/~srk31/research/papers/kell17some-p...
But certain languages (like C and C++, for instance) have built-in footguns. They were built in for various important reasons, but right now many of these reasons are not as pressing anymore. We can take something like Ada, Rust, OCaml instead, or at least use the extensive tooling that allows to statically check programs written in (a rather wide subset of) C and detect great many classical pitfalls.
So I'd say that it's not about safe languages, but mostly about safe methods of development for critical software, methods that remove large classes of defects that are commonly exploitable.
This, of course, can be only one layer of protection; the OS and the hardware design should also do their part. Advanced devices, like desktop and phone CPUs, have some very good hardware features for protection, like MMUs, w^x bits for RAM pages, secure enclaves, etc. Simpler devices (like IoT SoCs) often don't, and they can be potentially cracked into more easily.
Like the FAA does.
Never, ever going to happen in enterprise and consumer software.
Meanwhile, I have to argue with developers on a regular basis to convince them that they actually need to patch the libraries in their systems. Yes, even if they don't see an obvious exploit path. Yes, even if it's more work than drop-in-and-ship.
Getting them to use static analysis is a nightmare. Entirely too many developers view each finding as an attack on their style they can negotiate away.
As you so wisely and correctly say, we're at a point in time where we mostly have the tools to employ safer software development lifecycle methodologies. It is telling, then, how often we don't.
> There is no particular need to rewrite existing C code, provided the same benefit can be obtained more cheaply by alternative implementations of C
To be clear, those "alternative implementations" do not exist, and no-one actually working on C compilers or tools to make C code safer has been able to produce one, or even come up with a credible plan for producing one.
"Fail-Safe C" is a research project that has been dead for ten years. Note:
> Some benchmark results show that the execution time are around 3 to 5 times of the original, natively-compiled programs, in avarage
That overhead is actually a lot higher than similar projects I also consider failures, such as CCured.
To be clear, when we talk about "an alternative implementation of C", it needs to give similar performance and other properties (e.g. function and data interop) to other C compilers. You can't impose a 3-5x slowdown and say "look, C is safe".
When everything else fails, kill the bug with hardware memory tagging spray.
> To be clear, when we talk about "an alternative implementation of C", it needs to give similar performance and other properties (e.g. function and data interop) to other C compilers. You can't impose a 3-5x slowdown and say "look, C is safe".
Pretty much every language (except rust) which claims to be "C but safe" has the same issue, except you have to rewrite your program in it.
A C interpreter is not exactly the sort of thing that the embedded software industry has been waiting to deploy in production.
Further, CHERI is not enough to achieve temporal memory safety, it only provides the primitive that one could use to implement an efficient, correct tracing GC.
Perhaps most importantly, one can not use CHERI today. It is just not an option. Memory safe languages exist, and have existed for quite some time.
Pointer tagging and other approaches like CHERI are very promising. Keeping in mind of course that they double the size of pointers and have a global runtime performance cost. Less ambitious but otherwise similar mitigations are already being adopted.
(Disclaimer: been there, done that, part of the CHERI team)
Best of luck with the research, I'm quite bullish on the work.
I agree, though that's not intractable. There's security compliance like ISO27001, which can't say "you're secure", but can say "you've got a threat model and you take measures to address it". It's then something you document and provide evidence for as part of your compliance obligations.
If we applied the same thing to a programming language I think it could be very clear to see the differences between languages.
That's not at all to say that I think ISO27001 is a good model for security, just that we already have systems that handle very nuanced ideas, so a formal definition, or proof, etc, is not really as necessary as it may seem.
Maybe a safety ranking is in order? How many CVEs from [year], weighed by severity, are impossible to happen in [language]. Obviously this idea has serious problems, but you see my point: C would get a straight up zero, as it should! (well, not completely zero, as many vulns come from dependencies and C's complete lack of a package and dependency system is actually an advantage here)
Instead, people should be schooled to write better code. Thats it. Don't let some random new employee with no certifications write safety-critical code. Don't hire people who are under qualified. Its really that easy.
I have no idea, honestly, how you would introduce a use-after-free bug with C++'s smart pointers. Its the lack of schooling, the lack of certification, and the lack of a safety-focused selection of applicants, not a language that you can write bad code in if you ignore all warnings and advice.
Even rust has unsafe{}, but I dont see anyone complain when thats used to introduce safety issues, because "youre not supposed to do that, even though technically you could".
You can have all the safety mechanisms in a language, but you need control, too, and anyone who is completely unqualified will use that to break something.
I understand what you're saying, but I'm not sure I agree.
For example, look at Google Chrome. They've got mountains of cash. They've got loads of people working for them, and loads of job applicants if they want more. They've got a strong business case to work on security. They've got in-house pen testers, and a bug bounty program. They've got code reviews. They can afford any static analyser on the market. They've got sandboxing. They've got open source so many eyes can spot bugs easily. They can dictate terms on requirements - if Chrome vetos a new web standard and it's as good as dead, and if Google decides plugins have got to go, they go.
And they've got 177 CVEs so far in 2021 [1] - including such greatest hits as use-after-free, buffer overflows and out-of-bounds access.
You and I think we're writing secure C++ - but if the best-resourced team in the world can't write secure C++, isn't it more likely we're just fooling ourselves?
I might be wrong about this, too, but I think a lot of smaller companies have the really bad practices. Google shines as one of the major "suppliers" of C++ tooling, and their code quality is undeniably very high (considering the complexity of some of their codebases), but a lot of popular libraries are written by small teams, often not even under contract, and you end up with a lot of PR'ed stuff, which is great, but it needs to be thoroughly reviewed.
A good example of "unqualified" PR's was that recent one with the malicious PR into the linux kernel by some university. It was caught, but it does make you wonder how many vulnerabilities make it through because of simply trusting "random unqualified people" too much.
The point is that memory safety bugs are the gift that keeps on giving.
You can throw as much money into the problem as you want, and the inherent complexity of memory management means you are still gonna ship bugs of all severity.
That's needless extremism, moderate programmers benefit from safety mechanisms, and you can't hire a lot of perfect programmers.
As for the article, the dream to get all UBs right without a systematic approach is idealism.
Certainly C and C++ have more footguns than many other languages, but highly insecure as well as highly secure software gets written in all languages. I think coming up with better/easier avenues for digital-security-breach related lawsuits and fines is a better idea than banning specific languages.
And yes, for a really critical system I might consider taking something much simpler and potentially slower but formally proven correct, like seL4.
Google is working on Fuchsia as a new phone / chromebook OS, and it's very much focused on bulletproof security.
> Our kernel, Zircon, [is not in Rust](https://twitter.com/cpuGoogle/status/1397265884251525122). Not yet anyway. But it is in [a nice, lean subset of C++](https://fuchsia.dev/fuchsia-src/development/languages/c-cpp/...) which I consider a vast improvement over C.
https://blog.cr0.org/2021/06/a-few-thoughts-on-fuchsia-secur...
They are now starting to rewrite OS components in Rust, and I wouldn't be surprised if that happens to Zircon as well, it stared as a C anyway, and thus already suffered one rewrite.
A rewrite usually improves a lot of small things that were found in the previous version. It's like a new major version, even if it does not add new major features.
It's an expensive undertaking, though.
False. Which broken telephone did this come from? The Linux kernel has 0% Rust code in it and has no plans to change that.
(And no, "including experimental Rust support" isn't a plan to change that.)
You don't need Rust to compile the kernel. What's changed now is that you could theoretically write a kernel module in Rust. (But you can, e.g., write a kernel module in C++ too, I've done it before.)
One of seL4's points is that security can still be fast, no?
From https://docs.sel4.systems/projects/sel4/frequently-asked-que...
>To the best of our knowledge, seL4 is the world’s fastest microkernel on the supported processors, in terms of the usual ping-pong metric: the cost of a cross-address-space message-passing (IPC) operation.
But within the Linux kernel, you might not need that cross-process IPC at all, so if you're into squeezing every last microsecond of latency, you likely want your entire app running as a monolith in kernel mode.
But if you want security more than top speed, seL4 + a few daemons you write for other OS needs must be fine.
As for seL4, doesn’t that just support my point that highly secure software can be written in any language, even C? seL4 is primarily implemented in C. There “executable spec” is written in Haskell, but the actual kernel that people use is written primarily in C and a little Assembly.
Wow. It comes from the Libra hearing I guess? By any chance, do you have an link to the exact quote?
Edit: as hsod answered below (I don't really get why this answer was flagged) the congressman isn't really “complaining” and it's not about a Nigerian committing to Rust, but to Libra itself.
“I went ahead and did a GitHub search on who’s actually leading the development on the Libra core side and it looks like, I think it’s gonna be international because it looks like a native Nigerian that’s actually building the actual Libra core in front of the code development of Libra core” - congressman Denver Riggleman (R-VA)
He lost his seat in 2020
You define the threat model for your application and then justify why you're safe. In this case we'd be adding an explicit point to ensure that there are controls for attackers who can exploit memory safety issues.
You could end up with controls like:
1. We sandbox our code, so even though it's C we feel that we're safe
2. We use a memory safe language to reduce the risk of memory safety issues
3. We sanitizer inputs before providing them to the program
etc etc
What ends up being accepted as legitimate would be up to the auditors/ generally a matter of consensus.
Edit: And, of course, a lot of forced static typing and avoiding global scoped stuff. So, for example, you'd make a
struct Meter {
float value { 0.0f };
};
Meter wall_length;
Then even if you find issues in those third party code libraries, there is the issue of if you are allowed to change them yourself, or how those fixes get provided by the library vendor, warranties and so on.
What is this fascination with creating a new language every time we want to add some features? Is it the fame of creating and naming your own computer language? From a security perspective creating a new language reduces security because it will never have been as thoroughly vetted as the old languages.
Example: C++ shared_ptr gives you reference counting which helps prevent use-after-free errors. However, shared_ptr can't stop you getting a raw pointer or reference to the underlying object, which can then be used to trigger a use-after-free bug.
It's like saying "llvm is in C++, and llvm is used to compile Rust, therefor C++ is as safe as Rust". It's just not a tractable argument.
As for the rest of your post, sure, you can create a new revision of C/C++ that's memory safe - feel free to do so. It'll basically be a new language, but that's fine and has nothing to do with the parent's point, which is that we should incentivize memory safety - they didn't say it had to be a new language.
As for why people don't take that approach, it's because: it'll be a massive breaking change, so it's really no different from a completely new language. If you're going to create a completely new language you may as well take the time to fix a bunch of other issues while you're there.
edit: The above applies to libraries too. There are libraries that enforce stronger security in C++ (I want to say... IronC++? Something like that), and it basically ends up being, give or take, the cognitive overhead of a new language, but with the added bonus of no one actually using it.
> From a security perspective creating a new language reduces security because it will never have been as thoroughly vetted as the old languages.
This is unsubstantiated, and I highly doubt it is the case.
Would phones really be that high up given that in environments with high security standards usually people are not allowed to carry them? What about home appliances? Could a state actor hack a bunch of stoves and burn the houses down?
I can easily see some huge bureaucracy being put in place without much benefits ("Federal Programing Language Commission"?).
Most high security environments have a physical component, why not learn from there? Phones could just have a physical switch to cut off microphones and antennae for example, much easier to do than trying to police millions of lines of codes to be secure.
- Decoders and encoders for video, images, and audio - graphics libraries - Many parts of fast cryptographic libraries
This is typically for performance-related reasons.
Countries already do this. But they also put exemption clauses in the policies.
Ban exemptions (in all policies) first if you want to make progress.
But you won't like it.
It is the job of human intelligence to cause ‘discontention’ - discontent and contention. I like to think of how it as how you can make powerful gears almost seize if you understand their weaknesses properly.
Code reflects the programmer's understanding of the world. If this understanding is flawed, the logic will also be flawed.
Liability should be created where when you expose third party data. That would disincentivise data collection massively.
I can tell you exactly how this will end up: like PCI DSS.
That's billions of dollars that will get slowly steered in the right direction.
Software that can cause real damage to not just data and business activities, but endanger lives is completely legal to sell to despotic lunatics. If you sold a bag of fertilizer to a Syrian you'd go straight to Guantanamo, but somehow this is okay.
[1] https://en.wikipedia.org/wiki/Export_of_cryptography_from_th...
[2] https://web.archive.org/web/20051201184530/http://www.cyberl...
This begs the question: Is there a compelling reason why selling weapons to other states should be legal?
But apart from economic reasons, not really.
"Engineering" is a wide subject - the big stuff is carefully built and highly regulated - bridges and buildings. But as we go down the scale we see engineering give way to the problems of politics and money - tower blocks collapse for example, and then we see human level engineering - factory tools that try to meet inflicting goals, dangerous toys and so much more.
The software world should not beat itself up for not being like all those engineers - when lives are not on the line engineers get tied up just the same as the rest. And when lives are on the line, software and hardware engineering have learnt a few things - reduce the scope to the barest possible essentials - have a lot of redundancy and stick to well known designs.
If a car explodes because it got hit by an artillery shell, would anyone hold the automotive engineers responsible? If a building collapses because a bomb was dropped on it, would anyone hold the civil engineers responsible?
I think the difference here is that no car could be engineered to withstand an artillery shell, whereas we can imagine an iPhone not susceptable to this particular vulnerability (it already exists).
Perhaps one argument is that the space of _potential_ vulnerabilities in something as complex as an iPhone is so huge that it just isn't feasible to create one that can withstand all network-based attacks (in which the attacker hasn't obtained user consent, Apple's signing keys, etc)? However I'm not sure if I buy that argument, or not...
So, like all things, it's a bit of a matter of perspective.
I'm not really sure how to analogize this with software. The reality is some communications networks were just never meant to be secure. This isn't unique to the Internet. Nothing ever stopped anyone from tapping your phone and stealing your personal information that way except that it is illegal. On the other hand, a whole lot technical measures are in place to make sure it is very difficult and maybe impossible to "tap" a military or classified communications network at all. Nobody can stop you from intercepting radio, but good luck breaking the encryption.
But the national security infrastructure can't extend that level of protection to everyone, just as average citizens don't get police escorts and personal bodyguards assigned from the secret service. If someone wants to shoot you, the only thing the state does to stop them is make it illegal. Otherwise, it's on you to protect yourself, and we don't hold clothing manufacturers liable for not making your t-shirt bulletproof.
I heard that though it was often called the MIT bridge (because it connects MIT campus to Boston), they felt fine with the name “Harvard” after learning it was structurally unsound.
What about you - anything to back up your claim?
This work was mandated for projects that were, in aggregate, likely billions of dollars.
The cell phone's defenses are effectively 100% automated, and the attacker can buy a perfect copy of their intended target's phone to and safely try out an indefinite number of attacks on this copy until that attack is perfected. Once the attack is ready it can be deployed with very little exposure to the attackers. If the attack doesn't work, the attackers are unlikely to be discovered and you can make another attempt.
Attacking a subway system in comparison, the planning will be done based on diagrams and theory, the attackers only get one attempt, many of the defenses will be unknown, the target environment is highly unpredictable, includes human defenders, and even if successful the attackers will likely either die or spend the rest of their lives imprisoned.
I'm not a lawyer, but I don't see any language in the law that would exclude unsafe automated systems which are part of a building or structure.
Jean Ichbiah's team won this contract with the language 'Green' in 1979. They subsequently went on to further develop this and standardize it in what was then the Ada 83 language standard.
The more I think about the more it feels that Ada just came about to solve the right problem but at the wrong time.
https://en.wikipedia.org/wiki/Burroughs_large_systems
ESPOL/NEWP were the very first system programming languages to have UNSAFE code blocks, 10 years before C was even an idea.
Before that there was JOVIAL as well, https://en.wikipedia.org/wiki/JOVIAL
Like almost COTS(common of the shelf) Xeon, running some hypervisor and the 'legacy' within? Similar to what Symbolics did with Genera for Alpha?
edit:
[1] https://microsites.unisys.com/offerings/clearpath-forward
[2] https://docs.microsoft.com/en-us/azure/architecture/example-...
Azure?! Err...sure...
Secondly, having an issue with number 2 cloud provider at world scale?
Or would you rather have AWS?
I wouldn't trust that. The same way I wouldn't trust that from another, very established competing vendor, saying the same things, having the same nimbus of security and reliability.
Because sometimes there are blips on my radar, wherein some conference talk pops up, and people looked under the rugs of these systems, not even hard, just casually, and discovered some oopsies.
So it seems like that nimbus of security and reliability is more a result of very careful isolation from public networks on one hand, and inacessability for the bored and curious pranksters of the world on the other. But it is no absolute.
Especially if it's running in emulation on contemporary COTS hardware. (In the EFFING cloud!)
Just my not so humble opinion, though.
The most important fact related to his argument is one he doesn't even bother mentioning, namely that Android is mostly written in a memory safe language (Java). The thing he's asking for already exists and is deployed on most smartphones worldwide, yet all he has to say on the topic is this:
"While iPhones are more private by default and, occasionally, better-engineered from a security perspective than Google’s Android"
You cannot claim to be a spokesman for freedom, then demand memory-safe languages be used everywhere, and then praise the one system controlled exclusively by a single American firm that's written almost entirely in (Objective) C, a memory unsafe language.
That sentence is the only mention of Android in the entire article, the word "Java" doesn't appear anywhere and he seems to think that Rust is the only memory safe language in existence. Why should I care about this guy's opinions? Java has been drastically more successful than Rust when it comes to making software memory safe. Nobody is gonna choose to write the next AirBNB in Rust other than for fashion reasons, because it'd simply be too unproductive. Developers already complain about Swift and its horrible compile times, Rust would be even worse.
If Snowden really cares about this topic, he should brush up his Java skills, download some OpenJDK early access builds and start experimenting with writing video codecs and 3D engines using the new vectorization, memory span and value types features. Java is getting the capabilities to do even higher performance work traditionally dominated by C++, but in ways that preserve memory safety. The engineering is very difficult and it's unclear if Google will ever adopt it into ART, but it's there for them if they want it.
* All the UI libraries, networking APIs code.
* All the system apps and services like the home screen, the keyboard, the system server, the window manager, the telephony subsystem (very important!) and so on.
* Many of the system APIs including services like the alarm manager, dropbox manager, some cryptography services, location services etc.
* The entire developer toolchain: the build system and the IDE.
* All the client logic for Google Play Services, which is a big part of the overall Android API now.
* Large parts of the compatibility test suite.
* The standard libraries for all the above.
Just browse through the code and see for yourself: https://cs.android.com/android/platform/superproject
Most of the Java code is under the frameworks directory. No, not all Android code is written in Java. Lots of devs want to be able to use C++, and for some things its necessary. The point of the upgrade projects I just mentioned is that they are tackling the remaining cases where C++ can outcompete Java for things like media codecs. And the Java world has developed a JVM written in Java, actually several of them, so whilst Google hasn't done it, the tech is actually there. All this exists today, whereas an entire widely adopted consumer OS written fully in Rust is only a pipe dream.
Oh, and as for drivers, Google moved drivers out of the kernel and into user space some time ago (Project Treble):
https://source.android.com/devices/architecture
You can in fact now write drivers in Java. Google recommend you don't, but it's architecturally possible:
It is not GNU/Linux, which is what people think about when talking about Linux distributions.
Richard Stallman is known to throw a tantrum every time someone omits the "GNU" part but now, with Android, he has a point.
Targeting Android/Linux, if you so wish, it is no different than targeting Windows from GNU/Linux point of view.
I'd say the kernel, the drivers and the JVM are already enough to say that Android is not "mostly written in java". And that's not not to talk about whatever shit operating system is running on the modem.
Like farming, monocultures seem profitable, until the pests show up, and can swamp out all of the advantages.
No. The greatest danger is lack of software supply chain management followed by near-universal disrespect for formal complexity management methods.
The only way I have found to win at this "are we actually secure" game is to minimize the number of parties you have to trust. The smaller you get this figure, the easier it becomes to gain control over your circumstances again. How many of us can immediately state the exact number of unique parties that we have to trust as part of building solutions for other people?
What about complexity? Most of the time, something is insecure not because of malicious intent (covered by the trust angle above), but because its so goddamn complex that no one can say for sure if its correct or not. Why do we tolerate this?
When adding a dependency, knowing it exposes you to prosecution if it becomes a vector for security violation would give pause. A premium on attested-secure components might develop. If Facebook depended on Zst being secure to be able to stay in business, we might be more inclined to use Zst than e.g. unmaintained Zlib.
1. Developer has a problem to solve
2. Developer does a few web searches and finds libXyz, which looks like it solves the problem.
3. Developer tries libXyz and it solves the problem.
4. Yolo! A libXyz dependency is added, and now it's part of the product that the company stakes its name and reputation on.
Total madness. What else does libXyz do? What data does it collect? What does it do with that data? What bugs does libXyz have, and do they put our product at risk? What about security risks? Does libXyz have tests, and do they pass? How extensive is its test coverage, and would that be sufficient at our company? Does libXyz impact the overall performance of our product? What is its maximum memory footprint? Does libXyz limit our product to a particular architecture? What is libXyz's license, and is it compatible with our product? Who is responsible if libXyz fails and our company gets sued?
You're lucky if the developer even thinks about one or two of these, let alone fully auditing the library. Staking your product on the suitability of a library but not actually thoroughly vetting the library. And some products out there have hundreds of dependencies, all added in the manner described above. We're just handing out loaded guns.
Isn't this effectively the Microsoft/IBM/Oracle ecosystem play?
Thinking about this right now: annoying sure but why do you consider this pattern dark? I bet it reduces spam and trolling by orders of magnitude unlike say confusing cookie dialogs designed to make you surrender all your private info.
What's the purpose of these microphones? Do they pose more threat than the standard non-hidden microphone?
This strikes me as surprising. I have always been taught the opposite: if it feels good, it's probably bad for you, or illegal, or immoral, or all three.
Maybe I'm being too literal here.
As puritan as it feels, a large part of the population likes it that way and want it even more strict.
Pretty much. The only exceptions I’ve found are exercising and saunas.
I am unsure about his comments regarding unsafe code. Like many of the people have already stated here, that's a blurry line that has good intention but seems nearly impossible. I think more regulation and certification for both employees and companies is a much more likely to be successful.
We should at this point accept that this isn't going to change and move ahead with the belief that your data is already hacked and is not private anymore. We should discuss more on the exact consequences of this and take actions accordingly.
A mass movement of people can force a government to change.
Right now many western nations are involved in Pegasuses, what happens when China gets involved. We will have absolutely no control over what China does, especially when no country will publicly acknowledge that we are spying on people.
And it's not just fully interpreted languages - any language that manages memory for your and doesn't let you mess with it yourself (like Java, C#...) will be safer, assuming the compiler and runtime are secure.
1. Python is (usually) an interpreted language, and it's probably true to say that interpreted languages tend to have a lower attack surface (at the cost of lowered performance).
2. While Python is very popular in certain domains (numerical computing, ML etc), there are few low-level systems that are written in Python.
This makes it impossible to sandbox functions or imported modules, because they can communicate arbitrarily. But communication/access security is not the only problem. Resource security is something I haven't seen any significant language (besides Java perhaps) try to approach - being able to limit the memory and cpu usage of a (part of a) program. Modern languages should make it possible to do both with just a few lines of code.
I recommend reading about Capability Security[1], the E language[2] and the Principle of least Authority (POLA) [3]
[0] https://hforsten.com/redefining-the-number-2-in-python.html
[1] http://www.cap-lore.com/CapTheory/
https://en.m.wikipedia.org/wiki/Capability-based_security
[3] https://medium.com/agoric/pola-would-have-prevented-the-even...
These originated in the mainframe era, in a connected world before the internet, at the inception of the first multi-user systems: https://github.com/void4/notes/issues/41
This is a very interesting comment, and I’m going to study it and the links you gave. Thank you!
(Presumably an unusual thing to do -- I've seen libraries doing bytecode hacks but I'm not sure how popular any of them are.)
A subsequent import could patch the Python object namespace, but that’s normal and intended.
If a program has access to the byte code files, it presumable has access to the actual .py files, and can easily change them to do whatever mayhem it wants.
Isn’t that the opposite of secure? Pulling resource allocations out of the kernel and putting it into user land?
There is no issue with just limiting resources (unless there is unpredictable overhead). It doesn't have to be hardware resources either, it could be abstract/higher level resources like interpreter steps or managed memory slices.
I'm creating a series of VMs to show that this is possible, like rarVM, the recursively sandboxable virtual machine: https://esolangs.org/wiki/RarVM
Showcase: https://www.youtube.com/watch?v=MBymOp6bTII
When calling a function you can specify how many interpreter steps it can run until it aborts (and optionally gives you a continuation so you can "refill" and resume it later).
Stackless Python can do this too, but unfortunately due to the reasons discussed above will never be a safe language, also this specific mechanism works only in trusted environments since the called function has the ambient authority to increase its own resource limits: https://stackless.readthedocs.io/en/2.7-slp/library/stackles...
The most powerful point is to stop using unsafe programming languages. I usually use Common Lisp (for almost 40 years now), but I have been experimenting a lot with Swift over the last year. I have little experience with Rust.
I have seen comments that Swift and/or Rust are not appropriate for operating systems and general systems programming, but I call bullshit on that. The problem is a financial problem, expensive to pay for difficult to hire Rust and Swift developers to rewrite billions of lines of code. But it should be done. I pay an incredible amount of money to Apple every year and I expect it from them. Same for Microsoft customers (and Google).
See how much modern C++ you will find in AOSP and NDK, WinUI or XBox code samples.
Modern C++ exists only at talks in CppCon, C++Now.
Not sure I follow the logic here. The State has a monopoly on many things that are never allowed in the private sector -- violence being the most obvious one. We don't seem to have a huge problem with the split here; why we couldn't do the same for hacking?
What are the advantages of iOS's update model?
I hate iOS for it's walled garden bullshit but android seems to have found the one way to be worse.
even more so with Android & iOS.
Backdooring the modem firmware is far more useful, and reliable. Not to mention undetectable -- not only is the user unable to recompile or replace this firmware, they can't even get a checksum of it. Qualcomm's modem chips get their own private NAND flash that they can use as they please.
On the (very) few remaining phones that use two separate chips, what do you think the odds are that the host OS is hardened against attacks originating from its own modem? That is a hideously complicated protocol spoken between the modem and the host processor. Plenty of validation+overflow footguns. Finding exploits here isn't going to get security researchers promoted, if they can find them at all. "Exploit is available only to modem manufacturer" does not engender a high CVE score.
Oh, and, just to top it off, that interface between the modem and the host processor when they aren't on the same chip is... drum roll... USB. As in, BadUSB. As in, Mr. Phone says: "wow, somebody plugged in a USB keyboard! And a USB mouse!".
But hey, the modem doesn't even need to own the CPU. It can record and exfiltrate your GPS location quite happily via LTE, all on its own. And buffer up a nearly unlimited amount of location data on that private NAND flash it has in case you're out of cell range. All while pinky-swearing that location services are definitely absolutely turned off, promise. I don't know why people believe that location-tracking is ever turned off on a phone; it's just absurd to think that.
Race detection is good but can't eliminate races altogether.
Its type system only prevents data races for in-memory data structures, on the same OS process.
There are plenty of other concurrency races.
https://blog.stalkr.net/2015/04/golang-data-races-to-break-m...
Here you can find a POC.
You could argue that this is contrived, but it's also not going to be fixed. This is in contrast to Rust, where there may be implementation flaws leading to unsafety in safe code, but the language retains the right to fix those, even if it breaks code.
Still, I think there's really no question that Go is a major step up over C and C++.
If you can write in 100% safe Rust, Swift, C#, Java, etc. - and not use any of the unsafe escape hatches they provide at all - that would be best. But using some small amount of unsafety in any of those languages, or using Go, would still be a huge improvement over the typical C or C++ codebase.
So thanks for that. This marks the place where I acknowledge my mind was changed about the value of Rust.
[Update] Upon reading [1], I think that memory safety is still vital, but Rust has a very strange and cumbersome way of doing it.
[1] - https://doc.rust-lang.org/book/ch04-01-what-is-ownership.htm...
That sort of discussion is quickly dismissed on HN. And probably elsewhere on the web/over the internet.
Instead we frequently see discussion blaming users of the software, i.e., Microsoft's customers, or even suggestions to make the customer liable, or comments from "security experts" on how Microsoft has made such amazing strides in securing Windows (a tangent). In the real world, outside the Redmond/Silicon Valley monopoly space, how many mistakes does someone have to make before we start to suspect there might be problems with relying on that person's work. Even more, how many times do we hire someone knowing they have made 500+ mistakes in prior work leading up to their application.
If Microsoft products are so infallible when used as instructed by Microsoft, then why would Microsoft have a heart attack as Snowden suggests. What a remarkable state of affairs we have today where employers such as Microsoft can call their employees "engineers", and yet both the employer and employees are absconded from any liability for the so-called engineer's work. The number of "second chances" Microsoft gets is nothing short of astounding. A bit like the number of pardons we allow to Google or Facebook for privacy infractions. Infinite.
https://www.nspe.org/resources/professional-liability/liabil...
* you get paid a lot less
* the companies and industries move very slowly
* you spend a lot more time writing long-form, some time just re-using existing stuff wholesale, and almost no time building actually new things
I mean like Real Engineering fields. What we do in software is not real engineering, not even close. That has pros and cons.
I saw someone else talking about Rust, but I don't think that's what would happen in such a world. Rust is too new, if the company was actually liable for problems, the legal arm of the company wouldn't let you use it. I think what would happen is that everything would slow way down, half or more of the people working on code right now would lose their jobs, hobby programming would either disappear or become very insular and not well-distributed (because if companies are liable, then individual people will also get sued for bad software), and you'd spend most of your time working with small pieces of 30-40 year old technology.
I think that eventually, software may get to such a point. Just, be careful what you wish for.
The only reason our processes and practices aren't much heavier is because the stakes are lower. People do not die if a Tweet doesn't make it through, but they do if a bridge collapses through.
The threat model is also significantly different. If we go back to the bridge analogy, a company like Microsoft has to deal with tens of thousands of people trying to blow it up or find some weakness every day, while a million people are going over it. Just by sheer laws of scale they are going to have a tougher time, Real Engineering or not.
True, and what you're saying is generally true. But what were the total consequences of the Equifax breach? We can't even quantify it. Snowden himself in the article mentions activists and journalists being killed because of these vulnerabilities. There are definitely counterexamples.
I believe lightweight formal methods are quite promising and might let software move relatively quickly and economically while retaining some rigor. Look into Liquid Haskell for some ideas that might become mainstream.
I imagine a whole bunch of power could be derived from this in Haskell. I don't know how heavily contracts were/are used in Eiffel (I don't think it's so popular these days).
It would be great to know if contracts had a measurably useful outcome on projects and what that measurable was (addressed a market that we otherwise wouldn't have been able to, dropped runtime errors to only system based errors, etc).
WRT your projects, I imagine it would be great to see your product out there working with a known level of uptime and quality. I could imagine it's a bit of a shift from the normal "throw it together" of ... a lot of software.
I'd be curious to know what it's like to work on day to day, year to year. I imagine lots of software will still be "throw it together" for a long time yet, but even if the subsystems are formally verified, it could have a useful impact on the software that consumes these (verified) modules.
As for my day to day job, I think the real difference with common software development was that requirements were very well understood right from the beginning. This allowed us to formalize them into axioms and then derive the implementation in a stepwise fashion using some Standard ML tooling.
I've also worked on proving theorems on existing software artifacts, but that's way harder both in terms of effort and number of defects.
> * the companies and industries move very slowly
The reason the rest of us move so fast is we skip safety, security, quality, and maintainability to get to market. Those things are usually perceived "not bringing immediate customer value".
And deliver cheaper results.
And then the US would be forced to decide whether to accept imports of foreign devices and software (created under the no-liability framework) or to stay with homegrown technology frozen in time.
The best thing you can say about this proposed reform is that it would make a great plot for a sci-fi novel.
I mean the hub thing and synergistic collaboration are cool but employees are not 3x more productive because of it.
Why would it be frozen in time?
Local development would rendered unable to compete with the fast moving zero-liability model that quickly and cheaply delivered the features that consumers wanted. Either it's import would be banned or the local industry would crumble.
That assumption is deeply flawed. We do not hold toy car manufacturers to the same standards as actual car manufacturers. We do not hold every manufacturer of screws to the same standards as the manufacturers of screws on airplanes. Or rather, we do hold them to the same standards, just we know that certain use cases basically can not cause too much harm in the event of failure and thus in practice the standards needed to mitigate the worst case are much lower.
Software liability does not mean that everybody suddenly needs to take the same care as safety-critical industries. It only means that if you are making safety-critical software and you are incapable of separating the safety of the critical components from the non-critical components. What it really means is the repudiation of the one-size-fits-all lowest common denominator expectation of quality.
As for the rest of the open source ecosystem that goes into Linux (the operating system), it would probably be abandoned.
NotPetya which took down Maersk and did $300Mn of damages was apparently spread (through their Windows AD) by compromised admin accounts which they were lax at managing[1] rather than kernel exploits. The SolarWinds Orion security flaws were blamed on weak passwords, not OS kernel exploits. And if getting inside, something like last month's SystemD/polkit exploit[2] shows that attacking the kernel isn't always necessary for privilege escalation.
Linux the kernel is important but it's the heart inside the ribcage, not the first or last line of defense, or the main thing to target.
[1] https://gvnshtn.com/maersk-me-notpetya/
[2] https://github.blog/2021-06-10-privilege-escalation-polkit-r...
I imagine it would depend heavily on how large the liability was. I expect individual components would begin being replaced with "certified" commercial alternatives. If not existing ones, definitely new ones. Remember that they make money by selling to customers who would also be subject to the same rules. Look at healthcare, aviation, and finance for concrete examples of the effects (both negative and positive) that red tape has on software and IT policies.
There's an entire FOSS ecosystem and the vast majority of it is composed of small-ish slow moving projects. The tech industry is also an entire ecosystem full of small and medium sized players. Even if behemoths such as mainline Linux and AWS somehow survived unchanged I would expect a much greater chilling effect on smaller players that couldn't afford to take on such risks. New companies and software projects would become very difficult to get off the ground (healthcare is a good example here). With few to no new entrants forward progress would slow to an absolute crawl.
All of this has downstream effects. Fewer consumer devices running Linux would mean even less hardware support. Security related liabilities would almost certainly mean more vendor locked hardware. Would companies like Purism remain viable (or even legal)? The steady stream of new FOSS users and contributors would almost certainly dwindle.
Depending on how such regulation was written, could open source contributors themselves become liable for a freely provided product?
I worked at a place that had a formally verified application running on some mainframe. It was wonderful, except that the process was excruciating and maintaining that validation prevented any changes. Every code change cost a minimum of $25,000 2002 dollars.
It was dumb. They would have been better off with a paper process and army of clerks.
That's the funny thing. As a software engineer, if I have to build something that might kill someone, I just don't do it. But so-called "real engineers"? Bam, condo building down, people dead. Bridge down, people dead. Tacoma Narrows? Experimenting on people.
If "real engineers" were half as good at their job as I am at mine, we'd have a space elevator and people would be going up and down it every hour and the only problem they'd face is that the music is sometimes not that great. But they're too busy killing people to create $10 in value. I'm too busy not killing people and creating $1 million in value. The numbers don't lie, dude. The numbers don't lie. More value made. Fewer dead people.
People don't like to hear it because of the fetishization of this "Real Engineering" nonsense. But it's true. Software engineering's generational scandals are outdone by a "real engineering" project every day.
Maybe we should teach them ethics, because whatever class they took on that, it didn't take. As a practitioner of quality and making money with zero killing, I could help.
You're trying too hard. It's embarrassing.
> you get paid a lot less
Isn't that the point? That is, the argument is right now the money goes to the devs, management, and stockholders, when it rightfully should go to those people damaged by the software (or toward preventing them from being damaged).
> the companies and industries move very slowly
How much of this is due to liability law and how much is due to natural aspects of the relevant technology? Liability law may be part of the reason, but software probably naturally moves faster than other engineering fields.
If the money is supposed to go towards people preventing damage, wouldn't that include devs?
> How much of this is due to liability law and how much is due to natural aspects of the relevant technology?
I can only speak from my own experience in biotech, but moving slowly was due to a lot of compliance box-ticking that didn't actually contribute a lot to either safety, reducing defect rate or meeting requirements. Conway's law applied: since bio engineers and lab techs move slowly, so did the software org.
I agree.
Let's not use tools as a crutch. Roman engineers built bridges that are still standing today with little to no maintenance. There's no indication that those structures are going to fail anytime soon either. I think we can agree that their tooling was worse than our current bridge building tools.
> I mean like Real Engineering fields. What we do in software is not real engineering, not even close. That has pros and cons.
So the word engineer is probably loaded if not dated, a throwback to engineering's boom in the 19th and early 20th century. No idea why we still use it since software is more like math than anything else. Computer scientist just never stuck.
Roman engineers didn't understand the principles of civil engineering; I'm sure they built a lot of stuff that fell down (survivorship bias). So they started massively over-engineering their structures ("moar rocks!")
Survivorship bias is the favourite catchphrase for HN these last few years.
I think you overestimate the importance of academic theory over practicality.
"Moar rocks" is a perfectly reasonable way ro fortify a structure. Because 1000 years from now engineers will be wondering how any of our stuff survived ("moar cement and steel rods!")
So yeah, I have a healthy respect for practicality.
"Real Engineering" has definition and it fits SE, doesn't it?
Computer industry iterated and managed to reach the point where we can move really fast and do not break computers with bad code. That's result of thousands of hours of engineering effort of previous generations.
>Engineering is the use of scientific principles to design and build machines, structures, and other items, including bridges, tunnels, roads, vehicles, and buildings. The discipline of engineering encompasses a broad range of more specialized fields of engineering, each with a more specific emphasis on particular areas of applied mathematics, applied science, and types of application. See glossary of engineering.
There's a lot of legitimate criticisms about modern OS security, whether we're talking about Linux/Android, MacOS/IOS, or Windows. However, we can't ignore the scope of these programs. Supposedly Windows 10 is approximately 50 million lines of code and due to its overwhelming popularity it has almost certainly been targeted more often than all of the other OS's listed combined. I am pretty sure all of these operating systems are > 10 million lines of code.
Whose work are you going to rely on instead? The level of security in these OS's isn't equal across the board, but I assure you zero days exist for all of them and barring some kind of miraculous technological breakthrough, they'll continue to pop-up from time to time as long as they exist.
Suppose someone pulled off a miracle by making a security-focused OS that's easy for non-technical people to install and use and actually gains enough traction to establish a market share. If such a thing existed it would likely get lots of things right where others have failed, but they would also likely get lots of things wrong. It doesn't mean we shouldn't try and it doesn't mean we shouldn't encourage both old and new companies to try and improve the situation, it just means that its an incredibly difficult and likely never-ending task. Security is a process, not an achievement.
Probably not and they should try to reduce the size if it makes sense. However, if all of the OS's I listed are above 10 million lines even in the best case scenario a modern operating system isn't going to be anything less than an overwhelmingly large and complex program.
While i don't disagree with the point you're making entirely, I have made many mistakes in my career and i still get hired and i suspect you have too. Most people make mistakes, and still have jobs.
Realistically, people/orgs make mistakes. Requirements change. Expectations change, security practices change, etc.
Roman bridge engineers would likely struggle to build bridges at the scale and requirements they're built to in 2021 (miles long, tall, huge train weights, etc). Things change in technology jobs a LOT faster than typical NSPE engineering licensees' jobs. Its a different world. There are probably bridges under contruction today that were designed before the iphone.
Apply strict liability to software, and you'll see the same results. Every piece of software will have to be constructed with the care of a medical device. Expect most forms of technological progress to come to a halt. Some part of the HN crowd will post "I want that" from their iphone (which wouldn't exist under such a regulatory scheme).
There’s one story about a hip implant that went bad. Turns out the doctor recommending and performing the surgery was also the patent holder and had a vested interest in getting this particular implant in as many patients as possible. Turns out the patient was actually patient #8 who received this particular implant. Also the implant wasn’t fully approved yet and the FDA simply trusted the doctor to monitor the device for problems.
Also this isn’t isolated. The chapter has several examples of medical devices going into patients and patients experience negative health outcomes. Turns out laws are only as good as the agencies that enforce them.
If a company makes a bad implant, it's very visible, but all the potentially improved hip implants that never get built because of the barrier these laws create are invisible.
The point is there should be a better way that just pull the plug on anything potentially unsafe.
And then I can sue them to death or make a report to health authorities that will act accordingly.
In 2000, the average age of the nation's 150,000 single-engine fleet was more than 30 years. By 2020, the average age could approach 50 years
https://www.faa.gov/aircraft/air_cert/design_approvals/small...
Notice that "accident" definition conveniently excludes tons of partial failures! (see the legalese of 49 CFR § 830.2 - Definitions.)
To make parallel with broader discussion, the security failing of software could be considered "partial failures"...
Easy to say; not so easy to do. It is actually difficult to answer the [seemingly] simple question “What, exactly, is ‘Quality’?”.
I think I do a fairly good job, there, on my own work, but what works for me, won’t work for most.
Who even stands a chance? Colin Percival - math prodigy, cryptographer, former FreeBSD security officer - has paid out thousands in bounties for over two hundred bugs in Tarsnap software (not all security flaws) including buffer overflows, divide by zero, double-frees, mistakes in error handling, Unicode string handling error, numeric overflow mistakes, padding errors, user input handling bugs and one "critical security flaw" (against his very high standards) - https://www.tarsnap.com/bounty-winners.html
If someone like that working full time on a very constrained single purpose product can't make flawless software, what sense does it make for you to sneer at Microsoft as if Windows is singularly flawed here?
> "If Microsoft products are so infallible when used as instructed by Microsoft, then why would Microsoft have a heart attack as Snowden suggests."
They aren't, they're Swiss-cheese. So is approximately every other general purpose computing product, software and hardware, with the possible exception of a handful of very small very specific battle-hardened tools. Even the non-general-purpose walled garden appstore devices are Swiss-cheeses.
Possibly their heart attack would be because they are one of the biggest software development companies on the planet with a huge range of products used in tons of companies, so such a regulation would disproportionately affect them more than most companies? "All" Facebook develop is a web page, and very few companies use Facebook or Instagram or Oculus except by sending money and adverts to Facebook. Microsoft would have to secure a ton of large-scale software used on company premises in environments they have no say in. Seems to me the result of such a ruling would be something like Microsoft stopping selling on-premises software entirely and offering only web access to hosted Outlook, Office365, SharePoint, SQL, Biztools, Dynamics, with a ton of extra checks slowing them down, and companies faced with either using Microsoft online tools where Microsoft is responsible for the security or choosing on-premises tools like LibreOffice and Thunderbird and FireFox where they have to take responsibility, their legal and insurance would push them to Microsoft world with even less configurability or interconnectivity than there is now. It may be more secure, it doesn't sound great.
> "Instead we frequently see discussion blaming users of the software, i.e., Microsoft's customers, or even suggestions to make the customer liable, or comments from "security experts" on how Microsoft has made such amazing strides in securing Windows (a tangent)."
Not a tangent; Look at the recent discussion on HN about Windows Defender, and pretty much any security choice - it's full of people bemoaning Microsoft making decisions that are mildly inconvenient on the grounds of "how dare they think they know better than me", "I demand to be able to turn these security features off", "I want control to be able to do anything". If Adobe Reader is vulnerable and lets someone ransomware all your documents, what benefit is it to you if Windows underneath is an impregnable fortress? If someone can steal a user's 2-factor auth backup code from their house and access their remote VPN and exfiltrate data, what good is a perfect OS underneath doing for anyone?
This isn't whataboutism ("others are insecure so why can't Microsoft be"), and although it is somewhat in defense of Microsoft it's not "waah leave Microsoft alone", it's ... what kind of world are you living in where you heavily imply that Microsoft could have done differently and didn't? Microsoft trying to write Windows 3.1 in Pascal or ADA would have had their lunch eaten by every other company which didn't.
> "What a remarkable state of affairs we have today where employers such as Microsoft can call their employees "engineers", and yet both the employer and employees are absconded from any liability for the so-called engineer's work."
You think changing their job titles to "programmer" would improve the situation?
I think it's somewhat reasonable to complain about the approach taken to security in Windows. MS had a lot of work to do to stay on top as long as they did, but they were also extremely well resourced and dominant for a significant time, where they could have made bigger systemic changes to prevent swiss cheese getting released in the first place. They could've built their own rust-like language, maybe, built some great static analysis & fuzzing tools, or generally advanced their own internal exploit discovery to beat outsiders to the punch. That we've mostly settled for constant security updates upon exploit discovery in the wild and blindly assuming super old code is safe (until it isn't) seems like a failure to act (a failure of incentives?) and not a necessity.
They tried. Windows Longhorn was to be "entirely" managed code in the early 2000s, by 2004 they walked that back because vendors didn't want to rewrite their software for managed code and there were too many performance issues. Then they scrapped Longhorn altogether. As a research project they built: "Midori is an operating system that did not use Windows or Linux. Was written in C#. Took 7 years to build and included device drivers, web servers, libraries, compilers and garbage collection.". From what I know, they tried harder than Apple+macOS or anyone+Linux kernel and couldn't do it. To then casually say "they could've" is not so convincing. Especially if in this "you're responsible for bugs" world they would have had to have Proto-Rust production ready around the time of Windows NT in 1993 or so.
https://www.theregister.com/2005/05/26/dotnet_longhorn/
https://www.theregister.com/2004/05/06/microsoft_managed_cod...
https://www.infoq.com/presentations/csharp-systems-programmi...
https://news.ycombinator.com/item?id=27809296
> "That we've mostly settled for constant security updates upon exploit discovery in the wild and blindly assuming super old code is safe (until it isn't) seems like a failure to act (a failure of incentives?) and not a necessity."
The blogpost I link below[1] was written in 2004, and he says "I truly believe that the patching fad in which we are currently living is not going to last much longer. It can't. In another couple years, we'll have one full-time patcher to each system administrator. What's odd is that if companies simply exercised a bit of discipline, it wouldn't be necessary at all. Back in 1996 a buddy of mine and I set up a web server for a high-traffic significant target. It was not the Whitehouse; it was a porn site. We invested 8 hours (of our customer's money) writing a small web server daemon that knew how to serve up files, cache them, and virtualize filenames behind hashes. It ran chrooted on a version of UNIX that was very minimized and had code hacked right into the IP stack to toss traffic that was not TCP aimed at port 80. 10 years later, it's still working, has never been hacked, and has never been patched. If you compute the Return On Investment (Or ROI in the language of Prince Ciao) it's gigantic."
And yet the patching fad has got hugely worse since then, and all the factors he complains about - CEOs buying from salespeople, desire for customisability and flashiness - are all still driving the industry hard.
[1] http://www.ranum.com/security/computer_security/editorials/m...
Even when I say "they could've", it was in some part wishful thinking that we can even get to a state of the art where constant bug-patching can go away. I believe we can in theory. I also know that the nuance of reality often gets in the way of that kind of idealism.
Say goodbye to open source software!
Like one comment above said, there do exist ways to enforce that level of software security, (like railway traffic lights) but the cost would be ridiculously high, and those systems are probably not running any consumer kind of software stack, probably without an OS since Linux would has it own vulnerability as well. Those systems are probably made for custom hardware that the software vendor has total control of it as well.
For instance, if you like the Washington Post, and read this article you'll be quite annoyed at the "have to lose your spine" remark. The message is then diluted.
If you're just a regular consumer who likes the iPhone you'd be annoyed at the latte status symbol quip. The message is then diluted.
I don't think comparison with Linus is equivalent because he doesn't publicly advocate much whereas that's mainly what Snowden does.
(Perhaps I should have included this in the original comment.)
It's not about rights, it's about desires and personal insecurities. People who derive pleasure from condescension are just telling you that they're vulnerable to manipulation via flattery. It's bad opsec! :P