The urgent need for memory safety in software products
cisa.gov
cisa.gov
There is no regulatory body equivalent to the FTC or SEC or FDA for software, and it shows. Big software isn't magically more altruistic than big pharma or big agriculture or big finance. Given a choice between making huge amounts of money with very little overhead and raising overhead in order to protect consumers, they will consistently choose the former. It's high past time we have an agency with real teeth to regulate the software industry.
As long as you understand who will pay for it: you. Along with the rest of us.
This is the same argument against universal healthcare: “oh but look how much it will cost”. Yeah, it’s a lot. It’s also less than the price tag of the current system.
You people literally do not know what you're asking for, and you don't understand what would happen if you received it. When you stand on the shoulders of giants, it's bad form to kick the ladder away.
The real solution is to come up with a framework around data leaks and their disclosure with provisions for mitigations (à la GDPR, keep only the data you actually need).
Anything else will end up boiling down to "Java is the best language for everything" because Oracle spend $N on lobbying.*
*Replace Java/Oracle with another pair.
A sign of a responsible organization these days is how short-lived their hosts are. If every machine in an organization has an uptime less than a week, I feel pretty confident in that org’s ability to recover from a disaster. I also know it’s harder for attackers to keep a foothold once in the system, since everything is being constantly updated and reimaged.
But these days systems can be patched without restarting
So "uptime" is not necessarily accurate
Hazzard is a fictional county in Georgia that's the setting for the TV show “The Dukes of Hazzard”.
See the Boeing 737MAX fiasco for reference.
Regulations are pretty easy to circumvent with enough money it seems.
I was mentioning that it's a system as flawed as everything else if you just expect it to work.
Also I'd like this kind of direct life threatening decisions to become criminal charges (not just fines) for the ones signing them off, please, including in software regulations.
It's much harder to come up with the same kind of concrete regulations for software security (i.e., device must deliver no more than X dosage to patient). We might have to settle for forcing companies to carry exploit insurance, and make it easy to sue for breaches.
The software in one of the highest reliability industries is one of the highest reliability components. Unlike in other industries, especially consumer electronics, where even in low reliability products the software is almost always the lowest reliability component.
To put a more concrete take on this. In consumer systems, 5 9s is viewed as a gold standard. A level to aspire to, not necessarily one to achieve. In critical aerospace systems, 9 9s is viewed as the minimum, a level to never drop below. Many systems see 12 9s or higher.
The minimum standard in aerospace is 10,000x higher than the maximum standard in consumer. And the software reaches and exceeds that standard. That is how far the gap is and how well it works.
However, the downside is the expense and time. But, this can be mitigated by only applying these techniques where it matters. A key component of the DO-178 standard is identifying how critical each portion of the software is. You then only apply the most rigorous techniques for the highest criticality pieces. Of course, this requires the pieces to be well isolated, so that errors in one piece of non-critical code can not affect critical code; otherwise the non-critical code is actually, transitively, critical and thus demands increased rigor.
These techniques are tried and true in the most demanding circumstances. It would be prudent to start from these expensive working standards and work downwards to invent cheap working standards rather than fixing cheap non-working standards into cheap working standards. I think every software developer here can attest to it being much easier to start from something working, no matter how crude, than to start from something broken.
As for your actual point most systems would be better with a GC'd memory safe language then the hard real time those specs you mention are written from.
The DO-178 standards were instituted 30 years ago [1]. There are ~700 million passengers per year flying on commercial airlines in the US since 2003. Extrapolating that backwards to 1992, that would be ~20 billion service requests fulfilled. In 2019, the US averaged 1.57 trillion passenger miles over ~1 billion passengers resulting in a average flight of 1500 miles. That corresponds to around 3-4 hours on average at cruising speed. It takes around 3 minutes for a plane at cruising altitude to crash. So, since the DO-178 standards were introduced there have been 1.6 * 10^12 safety-critical service segments where a software failure over that period could cause a fatality.
There have been exactly zero fatalities connected to software errors since the DO-178 standard has been introduced. So, over 1.6 * 10^12 service segments every single one of them was successfully delivered by the software without failure. 12 9s.
Also, just in case it is not clear, calculating over multiple airframes does not result in redundancy reducing error rates. Every single airframe must succeed in their service of every single requested service. An analogous situation in IT would be having 10,000 servers with zero redundancy achieving 5 minutes of total downtime divided over all 10,000 servers resulting in 9 9s reliability.
Especially when some plane models need to be rebooted every couple of weeks because of software bugs ;)
https://www.theregister.com/2020/04/02/boeing_787_power_cycl...
If a major company got liquidated for extreme negligence, even once in a while, security would be much more in the forefront of executive's minds.
This is always brought up as a boogeyman every time regulation is proposed, but then we have stories about Incorporation by Reference [0], where Congress explicitly encourages agencies to incorporate standards written by industry groups into law. It's not a perfect system (for example, the copyright disputes), but it seems to work a hell of a lot better than the free-for-all we have right now in our industry.
> The real solution is to come up with a framework around data leaks and their disclosure with provisions for mitigations
Agreed. And an agency with teeth to enforce the framework. And the provisions for mitigations should have the force of law.
In other words: we need regulation!
Thankfully, it does not contain prescribed languages, just functionalities and things you have to do.
Conversely, we don't want to lose FOSS because single developers can't cope with the legal risks. Maybe there's opportunity for incentivizing cheap security certification of FOSS, or a CISA-approved subset of FOSS that is guaranteed to follow strict security practices.
What killed the shrink-wrapped market was first piracy and secondly the way people use their computers change. Then add on the easy sharing that e.g. Google docs enables and it's all over but the shouting.
The SAAS model also has the benefit of aligning value delivery with outlays temporally. No more wait a year before shipping: instead get quick iteration with an ongoing revenue stream. You can recover from missteps in a way that the old model could not.
Not sure how to respond to that. Emacs is around, the latter two I believe not, but approximately nobody is using them anyway. And I say that as someone who lives in Emacs.
> OS vendors still make money despite Linux and FreeBSD.
Apple makes money on hardware. Microsoft makes money on cloud, and has been desperately trying to turn Windows into a subscription service for a while now. They even non-humorously said exactly this in some message in (IIRC) installer or update screen - I remember Windows 10 telling me "Windows is a service" as if that was a good thing.
> The GIMP didn't kill Adobe.
No, but again, GIMP and Krita are used by approximately no one, meanwhile Adobe switched to a quite abusive subscription model.
> And if you want to sell proprietary libraries your market is going to be limited because OSs will distribute them and it will be hard to keep up when customers are fixing their own problems.
I don't see how. Think about the 100 (or 10 000, if you're doing webdev) dependencies your product uses. Most of those could be a licensed product, creating a cottage industry of small vendors making a living from developing specialized libraries. This thing is real in some areas, e.g. industrial software, where there are e.g. protocols that are too complex and boring for Open Source community to care about supporting in full. But it's something impossible in general, because popularity of open source simply prevents this kind of market from forming.
Now, to be clear: I don't have a strong opinion on whether this is good or bad. Still trying to fully understand that one. But what I know for sure is that open source is a huge disruption to how the market would've worked otherwise.
> What killed the shrink-wrapped market was first piracy
I'm not convinced about that. Maybe video games had a problem, but money in software in general was in B2B, not B2C, and there the piracy was effectively mitigated by the legal system. In fact, both Microsoft and Adobe used piracy as opportunity because of that - kids would grow up using pirated copies of professional software, and then as adults, their companies and their employers would have to buy the properly licensed versions of that same software, because that's the only thing anyone has experience in using.
> The SAAS model also has the benefit of aligning value delivery with outlays temporally. No more wait a year before shipping: instead get quick iteration with an ongoing revenue stream. You can recover from missteps in a way that the old model could not.
SaaS has some benefits, but they tend to be single-sided, unless the customer and the vendor are comparable in power, which usually isn't the case. So while e.g. IBM or Microsoft can force Google or Adobe to behave, my small business will be taken advantage of by large SaaS vendors.
Computers always allowed you to copy code for free and any new concept that took six months to be made up, can be duplicated for free in one second.
The real problem is that you have to ship code that can be disassembled and read by another human being, to sell software. This fact annihlates any incentive and control an author-of-software has over their conceptual labor.
What we need is a method to sell a binary (and source-code) that are completely free of human-perceivable information AND authorized against a purchase server. Identify the code author or not, both are fine, what matters is the conceptual labor is protected.
That's it. We do that and it becomes viable to do hard mental work for a long time, and take risks on bold moves. Make the concept work and sell the baked end result without giving away the recipe.
Every other industry keeps the means of production to themselves.
But for some awful reason, programmers are expected/required to ship the means of production, the understanding of the code and the concepts of the code AND be ready to answer anything, in a relationship with the purchaser.
The balance of effort between software developers and the benefit everyone else gets from them, is completely messed up.
That sounds impossible in principle, not without taking away ownership of the machine from the user. I guess TPM + homomorphic encryption might allow this one day, but that doesn't feel like a win for society.
> But for some awful reason, programmers are expected/required to ship the means of production
Pedantic, perhaps, but I wouldn't classify code as "means of production". Dev tools and supporting tooling, maybe, which no one is required to ship.
I sort of agree with you're saying overall - you're pointing out the fundamental issue, open source is just something enabled by it. I focused on open source because, arguably, this reality of "messed up balance" was still manageable through the legal system, and it's the FLOSS becoming a cultural phenomenon in the industry that broke this.
Generally it does not:
Microsoft Windows EULA states:
"Disclaimer. Neither Microsoft, nor the device manufacturer or installer, gives any other express warranties, guarantees, or conditions. Microsoft and the device manufacturer and installer exclude all implied warranties and conditions, including those of merchantability, fitness for a particular purpose, and non-infringement. If your local law does not allow the exclusion of implied warranties, then any implied warranties, guarantees, or conditions last only during the term of the limited warranty and are limited as much as your local law allows. If your local law requires a longer limited warranty term, despite this agreement, then that longer term will apply, but you can recover only the remedies this agreement allows."
Source: https://www.microsoft.com/en-us/Useterms/Retail/Windows/10/U...
Oracle EULA states:
"Disclaimer of Warranties; Limitation of Liability THE PROGRAMS ARE PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND. ORACLE FURTHER DISCLAIMS ALL WARRANTIES, EXPRESS AND IMPLIED, INCLUDING WITHOUT LIMITATION, ANY IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NONINFRINGEMENT."
Source: https://www.oracle.com/downloads/licenses/standard-license.h...
And that's a real damn shame, too. If only we had actual Engineering certifications for software engineers.
Things would be a lot more expensive... but perhaps they'd be a lot more consumer-friendly too.
See Canada: Software Engineers are called Software Developers there, problem solved.
(Now, there might be an argument you could make that, if software engineers had actual certifications, they would have the leverage to push back against Microsoft management and say "no, we're not going to ship that". But I'm pretty sure that wouldn't work, at least not at this stage of the game. Microsoft would just find coders in India or the Philippines or somewhere to implement it where there are no such professional regulations. No, when a company the size of Microsoft does something like that, the problem is not with the individual contributors.)
An ethics course is no panacea. But what I learned in ethics has been far more useful in my career than a lot of my hard CS subjects.
I mean, you don’t sign a 20 page contract when you visit a restaurant saying you agree not to sue them if you get food poisoning. Or sign something saying you take responsibility if you die in a plane crash, or from a shoddy building collapsing on your head. Companies in all other areas of society are held responsible for foreseeable harm caused by the products they sell. EULAs make a mockery of that, and the law should void or ban them.
Software also suffers of untold expectations. Companies regularly spend as much time writing specs as developing the real software, and yet, imprecision still dominates the implementation.
In fact, in the US, if you get a licensed engineer to do an inspection or design job, they will also often include a list of disclaimers to avoid liability. You often have to negotiate the lack of said liability waiver in advance for a higher fee. Note, liability waivers don't protect from negilence, just all other scenarios.
That's a deep rabbit hole that leads to the Underdark (https://en.wikipedia.org/wiki/Underdark) of Enterprise Legal Agreements.
Only high-level legal wizards and magisters can go there and expect to understand anything and keep their sanity.
Not a lawyer, but I expect it comes down to "if you follow our guidelines and pay us enough we can make the following guarantees, but there are certain things that we simply do not cover."
This is exactly what a warranty is though.
Yes, but it is not for consumers. It is for big businesses that pay millions and have specific expectations. The more you are willing to pay to more you can get within specific limits.
Your original statement, "If you pay for software it generally comes with some warranty and guarantee of fitness for purpose." does not hold for consumers and only partially holds for enterprise customers.
It might be better stated: "If your business / entity / government has specific needs (e.g. compliance with legal or medical regulations, industry certifications, ISO requirements, auditing, etc.), has a large sack of money to pay, and a team of lawyers, then there are some software publishers who will sign contracts to guarantee you certain qualities about the software you license with them and perhaps some level of support of that software."
---
Limited Remedy. If Microsoft, or the device manufacturer or installer, breaches its limited warranty, it will, at its election, either: (i) repair or replace the software at no charge, or (ii) accept return of the software (or at its election the device on which the software was preinstalled) for a refund of the amount paid, if any.
---
Unless it's some sort of bespoke software with a detailed contract (the details of which aren't available), I don't believe that's true.
Every single software license (as well as websites and other services) I've ever read (over the past 30 years or so) has had some similar disclaimer.
Would you provide an example that doesn't follow that pattern?
What needs to be done in commercial software, as is done in all other industries and in existing safety-critical software deployments, is segregating use cases by criticality and requiring deployments reach and guarantee the necessary quality level for that criticality class. It is the job and liability of the deployer to guarantee the requirements are met. They may push these requirements down to their suppliers, but it is the deployer that is ultimately responsible.
This is why Home Depot is not required to only sell aircraft-grade rivets. It is the airplane manufacturer’s job to source adequate components, not the job of all component suppliers to guarantee their components are fit for all possible purposes.
Under this model you can release any software you want. It is the manufacturer using the software that bears the burden of proving it is adequate. Or, if the software maker want to voluntarily take on extra burden, the software maker can verify and certify that the product is adequate for whatever purpose they wish to support.
Quality where it is necessary, and flexibility where it is not.
People lost their shit over the 737MAX. Boeing lost a ton of money and credibility. It made everyone realize that strong regulations need to be maintained and how it's a bad idea to allow manufactures to review their own product. It's also likely to result in even more regulations and a executives at Boeing being criminally tried.
Regulations are incredibly effective, but you can't go soft because they are working. Regulators have to be ever vigilant and politicians who push for softening of regulations need to be voted out of office. We are finally starting to see that conspiracies to skirt regulations constitute criminal behavior by senior company leadership, this is a good thing for the public.
I'm not sure I really see a distinction between that and a hardline tough on crime position. Not all laws/regulations are good and even an ensemble of individually good rules could be problematic in aggregate. Though in the current political climate, I would tend to agree that those pushing for reducing regulation are mostly not doing so as part of a well-intentioned effort to reduce adverse effects of regulation for the public good.
So now you want to regulate in security, for a device that was never meant to be secure in the first place?!
This security business is a racket. If you wanted computers to be secure, you would have designed and sold ASICs from day one.
You can't regulate software without also regulating the hardware.
The UK locks down privacy laws and now you guys want to "secure and regulate software".
The internet, no more. Offline is the only way to program, and keep software in-house.
> The software industry must not kick the can down the road another decade through inaction.
The software industry most assuredly will, unless legislatively forced or otherwise compelled by the industrial environment.
This liability is already non-zero. I wonder if the penalty fees are too low, or if there's some kind of legal shield that reduces the risk
Software can be installed on billions of devices. Even a low liability could be financially ruinous...
Especially for open source.
At least a legal framework will tell you what you should and shouldn't be doing.
Also outside computing industry everyone is liable on the eyes of the law, even if doing charity.
If the user has accepted such provision before making the binding acquisition of the said product, then it is another matter.
The DoD funded Ada which curiously is not mentioned here.
The above sentence is also strange because it is not "the market" that "tolerates" dangers to the user (customer). It is the legislature that tolerates it. If it's legal and triggers no civil liability for a manufacturer to "sell" (a license to) a product that is dangerous, then why would it refrain from doing so. And further what if the customer has no viable alternative to "purchasing" (a license to) the dangerous product.
Sell and purchase are in quotes in the event we are referring to either (a) so-called "tech" companies that derive necessary revenue not from software (license) sales but from advertising services sales and/or (b) software companies that sell licenses to OEMs (customers) rather than directly to users. In neither case does the user actually purchase the software (license). In these cases, even if we assume, correctly or incorrectly, that users are "the market", it is arguable they have no meaningful choice of viable alternatives. This might explain why they "tolerate" dangerous software. If we assume, correctly or incorrectly, that advertisers or OEMs are "the market" then, as above, they "tolerate" dangerous software because legislatures have not made this illegal or a source of civil liability.
As an example Red Hat OpenShift added support for crun(https://github.com/containers/crun) this year(https://cloud.redhat.com/blog/whats-new-in-red-hat-openshift...), which is written in C as an alternative to runc, which is written in Go(https://github.com/opencontainers/runc)...
Specifically, OpenShift is supported on both POWER and IBM Z in addition to x86 and ARM.
But probably the main reason is that the project started in 2018, when Rust didn't have nearly as much momentum.
> OpenShift Container Platform uses CRI-O as the container engine and runC or crun as the container runtime. ->The default container runtime is runC.<- Both container runtimes adhere to the Open Container Initiative (OCI) runtime specifications.
As for Rust-written runtimes, there is another project in the Containers GitHub namespace called youki[1] that is attempting to do just that.
[0]: https://docs.openshift.com/container-platform/4.13/post_inst...
That being said, I'm still puzzled why email programs don't differentiate mail from your bank vs. mail from yourbank.phishing.site.
In the same time I am teaching my daughter how to do buffer overflows with a board game I just released: https://punkx.org/overflow/
I think memory unsafety is what makes computers fun, ever since I read phrack 49/14 it made me see programs in a different light and actually enjoy programming way more.
In the same time I would like to not have zero click zero day exploits on my iOS phone where I cant control at all what code is executed.
(Anyway, happy coding…week…or whatever.)
Which isn't to say we can't take some learnings from the idea, just that it isn't simply a matter of money.
Most web services do this in the network level with load balancers that forward only a single port to the service VM that is otherwise behind NAT to prevent any access to any other program in the VM that is left listening on a port. You can extend this concept to the input and output processing of the service.
so many critical exploits use the same characters and lengths as intended inputs. Also, if firewalls were a replacement for secure code, no one would be talking about memory safety.
https://github.com/stong/how-to-exploit-a-double-free
The analogy to firewalls is that you would specify the exact condition of the input for it to forward to the actual program. For example, if your endpoint receives json, you would validate the json and check each field value for valid range, ie min max number of characters and what those character values could be for each field. Just like a firewall limits who can talk to who in way.
You should be able to run any program you want, no matter who wrote it, nor the language used, safely. If you can't do that, your operating system sucks. Security is the operating system's job.
Just because we're collectively hotwiring everything to the computing grid, and just telling programmers, admins and users to be more careful, doesn't mean its a sane way to live.
To cope with our current insane situation, we've all discovered coping mechanisms, like putting things in VMs, Containers, using Sandboxes, etc... none of them are as effective as specifying at runtime exactly what side effects are allowed. The user can do it using powerboxes, the admin can do it in a batch file.
We can have easy to use, secure, general purpose computing, but at present, the Overton window is too far away for most people to see it as even possible.
I supposed it was like that with the electric grid at some point, too.
That’s just flat out not true. It’s jot possible for an operating system to be perfectly secure without unduly locking down the capabilities of applications.
Considering the enormous feature-set of 1981's Intel iAPX 432, perhaps in 2023+ it should be the job of the hardware instead.
Lisp Machine 2.0?
(looking for real-world anecdotes, not preexisting assumptions)
Transportation. Motor vehicles are a huge chunk of the market despite the existence of less deadly alternatives.
It was never a good idea to use a performance-oriented language for software that should have had its highest priority as security, and it will be remembered in the long tail of history in the same way that asbestos is: a hazardous material that took us a long time to rid ourselves of.
You sound like you're using the uncomfortableness as evidence of the truthfulness. Logical inference doesn't work like that.
Particularly his wording: dereliction of duty
There's pretty much no reason to not use a memory safe, performant language like Go at this point for infrastructure-ish software.
When they totally could be doing `go get sources.redhat.com...`
If someone works around that on a developer machine, it shouldn't matter, because release builds should still come from an automated system, right?
You say "just 8 years ago" but another way of phrasing that would be "only 4 months after Rust went 1.0". And that's the first release, which means work probably started before Rust went 1.0
>Red Hat is one of the companies that also has a strange C addiction for almost anything.
This is kinda true and I wish it was different, but most of the examples people bring up are ones like the one you brought up. Rust wasn't a "safe choice" at the time they were started.
There's other factors too, like needing to support POWER and Z architectures. If you have a lot of such products written in C, and you have a GCC toolchain team in-house, that's not so difficult. If you have a lot of such products written in Rust, and your LLVM toolchain team is much smaller and the compiler itself less mature on those platforms, it's a bit more challenging. And then the whole issue of dynamic vs. static linking, where there's a ton of organizational inertia in favor of the former.
All that said I think the list of valid excuses is increasingly small nowadays.
Agree. However, we need to solve the problem from the supply side (create more and better Rusts, Gos, Zigs, etc. so that people choose to use them), not from the demand side (shame people into adopting Rust).
I think there is a significant portion of developers that want to leave C++ for a better language, but for different reasons may not be satisfied with Rust or Go, so they just stay on C++.
1000 C++ servers vs. 100,000 Python servers is a big reason to avoid Python.
To keep iPhone apps responsive, they saw problems with apps using GC.
They removed tracing GC from Objective-C, because it was a failure when dealing with C semantics.
Automating retain/release calls was a much easier and safer approach, hence plan B.
In any case, reference counting is a GC algorithm.
Another may be that it simplifies the runtime for system code such as kernels and device drivers.
But people do use GC on their phones, insofar as they like to use web browsers.
.NET interop with COM is a good example of how mixing a tracing GC with a reference counted runtime requires an higher engineering budget that Swift team most likely wasn't willing to spend resources on.
It has been proven multiple times that reference counting is slower and has performance issues when dealing with large graphs, and also stop-the-world issues when cascade deletions occur.
If only pro-RC folks would bother to read CS papers and benchmarks.
I can think of two reasons for people not moving: 1. inertia, aka "not going out of comfort zone to learn new things" ("learning" includes understanding why things are different, not merely knowing they are). This won't be addressed by another new language that people will also need to learn.
2. Existing codebases won't magically turn from C++ to Rust. Legacy is actually a good reason to still be using C++ today, but another language also won't address this, as the fundamental issue is you can't change the existing code to be safer without effort.
In both cases, time and effort on the language best positioned to replace C++ (Rust) will alleviate these issues. In particular this effort could be spent to fully close the feature gap with C++ (specialization, more expressive consts, ...) and improving Rust/C++ interoperability.
No need to "shame" people, but finding a way to replace C++ use by Rust use is how we will solve the problem.
All of those are currently at the research project stage, so Rust is really the only game in town to replace C++ at the moment.
This has largely happened already. Java is in fact a memory safe language which has replaced a whole lot of C++ code since the late 1990s. It's only in a few highly performance-oriented areas where the debate ("should we be replacing C++ with a memory safe language?") is still undecided.
CVE-2017-5638 was famously exploited to gain access to Equifax's network and cause a massive data breach. Log4Shell is continuing to cause problems in all kinds of networks due to the extremely widespread use of the Log4j library. Java is definitely an upgrade over C/C++, but there are definitely still security risks.
C++ continues to be popular because it's a powerful language and can easily integrate with a ton of libraries, and most people don't care much about security.
My gut feeling is that a different language with less friction will gain more traction.
But hardly any languages integrate with C++ very well. Have you ever used SWIG? (Shudder...)
I do agree it would be nice if it could. The cxx crate is ok but we could do a lot more.
Maybe something like SWIG but based on LLVM so it actually works properly.
It will take generations, doesn't make it impossible, and you already see this change happening in mobile OSes like Android, where C and C++ are frowned upon for regular app development other than games.
"C and C++": 1 mention
"Rust": 1 mention
They only mentioned Rust once as an example of a "memory safe" language, but did not elaborate as much as they pointed their finger at "C/C++". This is bad and unconvincing writing.