Standards for Software Liability: Jim Dempsey, Lawfare, UC Berkeley Law
lawfaremedia.org
lawfaremedia.org
However, we don't have the time to let a software liability common law regime develop over decades, given the threat presented now. Therefore we should adopt the products liability, design defect analysis, which he explains as:
In fact, it turns on two reasonableness inquiries: Was there a “reasonable” alternative design that the manufacturer could have used to avoid the vulnerability, and, without that alternative, was the actual design used “unreasonably dangerous”?
The author's summary liability regime, in light of the NIST guidance, would be as follows:
1. A rules-based approach would define a floor—the minimum legal standard of care for software—focused on specific product features or behaviors to be included or avoided.
2. However, a list of known coding weaknesses cannot suffice alone. Software is so complex and dynamic that a liability regime also needs to cover design flaws that are not so easily boiled down. For these, I propose a standard based on the defects analysis common to products liability law.
3. But this liability should not be unlimited or unpredictable. As the Biden administration’s National Cybersecurity Strategy recognizes, developers deserve a safe harbor that shields them from liability for hard-to-detect flaws above the floor. For that, I would turn to a set of robust coding practices.In general, yes.
However, I hate Rust; my software wouldn't exist if I had to write it in Rust. [1]
So I hope there would be a way for me to justify using C, along with a plan to minimize memory vulnerabilities.
In my case, my plan is to fuzz, use every one of the fuzz paths found as a test case, and ensure all of those test cases come out clean in sanitizers and Valgrind.
Oh, and I'll also do error injection, like SQLite [2], to exercise even more paths, and those paths will also have to come out clean.
[1]: https://gavinhoward.com/2023/02/why-i-use-c-when-i-believe-i...
This included an RFI for which there were many responses, including a number of major tech companies (Microsoft, Google, others). The general consensus supports a move to memory safe languages, BUT many of the respondents also caution about the need to prioritize and carefully manage such a transition, and the need to continue to support and assure code written in memory unsafe languages using other assurance techniques like fuzzing, integration testing, unit testing, static code analysis, and more.
Hopefully that helps explain where things stand at the moment regarding at least US movement on this topic, and also to assuage concerns about a unilateral mandate that things get rewritten in Rust (or another language).
[1]: https://www.whitehouse.gov/oncd/briefing-room/2023/08/10/fac...
I also know that despite some having caution, there are those that would speed ahead with RIIR.
If the desire is -- you would like to create new software in C because you like C, I think that presents some really difficult questions in this framework, simply because a reasonable alternative exists.
First, I think you have to ask: What kind of programs at you writing? Are they FOSS? Do you sell them? Are they likely to cause "damages"?
If what you are doing is writing software for the joy of writing software, I'm not sure you have much to worry about. If you're writing a bank transaction server, exposed to untrusted inputs, in C and you're selling this software, you might have something to worry about, because there exists a reasonable alternative in Rust.
Rust is unfortunately not a reasonable alternative in some cases.
As an example, my software needs just a C compiler to build itself. You can build my software completely with:
tcc -run bootstrap/boostrap.c tcc
Another reason Rust may not be reasonable is the pervasiveness of async. Personally (not in general), async does not work for me.
Rust is not the last word in programming languages. I am working on an alternative that will not only not have async, but will have more static analysis because destructors will be guaranteed to run.
If you had read my post, you would know that I am planning to rewrite my stuff in that language as soon as possible.
If Rust becomes required, I would be forced to stop work on that language, and progress would slow down.
There are valid reasons to use C, even for software that handles untrusted inputs.
Sure, and tort law is a flexible framework, so you will be allowed to make that argument. The only problem is if your software blows up in a way Rust was meant to prevent.
I am trying to hit the same level of robustness as those, and if their level is still not acceptable, our industry will implode.
I get that it's not okay to have 70% of vulnerabilities be memory safety bugs, but C projects shouldn't have to be perfect on that front. Just close enough. Maybe 10% or less.
If your software blows up, in a way Rust would have prevented, and caused damages, isn't it kind of fair someone say "Hey maybe you shouldn't be using C"?
My point was simply -- if your software is not in a place where damages occur, fine, who cares, write it in C. But if it is, and Rust exists, and could work just as well in that environment? And if you must still write in C, that's still fine, until you start selling software and causing your customers damages.
> But C projects shouldn't have to be perfect on that front. Just close enough. Maybe 10% or less.
Why?
No, that would not automatically be fair. It depends on context.
> But if it is, and Rust exists, and could work just as well in that environment?
But that's what I am saying: Rust won't work as well in that environment.
My stretch goal is to have my build system replace the awful custom Makefile build for FreeBSD. In that context, Rust doesn't exist, even if they add it to the base system because something would have to build Rust and LLVM.
Same reason my bc is in C: it is used to build the Linux kernel.
Anyway, who defines if Rust works "just as well" in an environment? It all depends on context. You can't say that any Tier 1 and Tier 2 platform must use Rust for reasons such as the one I laid out above.
And what if my language succeeds and has more bug smashing power than Rust? Should the directive immediately be, "If any Rust project causes damages that Yao could have prevented, that project owner is liable"?
Of course not.
> > But C projects shouldn't have to be perfect on that front. Just close enough. Maybe 10% or less.
> Why?
Call it a hunch, but roughly, I think that a C project that reduces its memory bugs to that much would have fewer vulnerabilities than an equivalent Rust project, just because it would smash a lot of non-memory bugs along the way.
That is the criteria that should be used: how many bugs does it have per KLoC, number of statements, cyclomatic complexity, whatever.
Otherwise, you're just effectively mandating Rust anyway, and Rust projects could get away with murder.
But set an objective criteria like that, and SQLite and Curl would survive, I would make my project hit it, and poorly done Rust projects would have to clean up their act.
IOW, I do agree that Rust should be the default, but we shouldn't choose a metric that is clearly biased in its favor.
Well then the problem would be different?! If instead the problem is the problem as I described it, you have an issue!
> Anyway, who defines if Rust works "just as well" in an environment?
Dude, you're super angry about Rust and you are refusing to address how any of this might work in a products liability context.
If you want to say software development is so inherently creative, and subjective, it can never have standards, standards which the rest of the world relies upon to build cars and toys and industrial boilers, then good luck with that opinion.
> Call it a hunch, but roughly, I think that a C project that reduces its memory bugs to that much would have fewer vulnerabilities than an equivalent Rust project, just because it would smash a lot of non-memory bugs along the way.
Good luck making this argument!
> IOW, I do agree that Rust should be the default, but we shouldn't choose a metric that is clearly biased in its favor.
Unfortunate for you, products liability is actually about that metric: safety. If you want to forgo better practices, languages, etc., that's fine. Because products liability law could never simply mandate Rust. You must only be reasonable in your decision making process.
Maybe. Others will have different situations, so no, even if the problem is as you described it, you may have left out some variable that precludes Rust.
> you are refusing to address how any of this might work in a products liability context.
Yes, I did. I said that we need some objective measure, and I gave a few examples. Anything that doesn't measure up gets hit with liability.
> If you want to say software development is so inherently creative, and subjective, it can never have standards, standards which the rest of the world relies upon to build cars and toys and industrial boilers, then good luck with that opinion.
I never said we could not have standards. That is the opposite of what I said.
I said that we should have a fair objective measure as a standard, not "no memory bugs" or "always write in Rust on Tier 1 and 2 platforms."
> Good luck making this argument!
It will actually be easier with time as more Rust projects appear.
But as a taste, I find it illuminating that, despite calls to rewrite SQLite and Curl in Rust, no one making those calls has actually sat down to do it.
And I do have some personal data in that regard. [1]
> Unfortunate for you, products liability is actually about that metric: safety. If you want to forgo better practices, languages, etc., that's fine. Because products liability law could never simply mandate Rust. You must only be reasonable in your decision making process.
Yes, I agree.
But if we choose the wrong standard, then products liability law will effectively mandate Rust, and that would not be good.
And "safety" is not just about memory safety; it's about that and everything else. Rust is not a valid excuse to forget everything else.
[1]: https://gavinhoward.com/2023/12/am-i-a-good-c-programmer/
[1]: https://www.cl.cam.ac.uk/research/security/ctsrd/cheri/
If someone paid me to, I'd do it in a heartbeat.
For CHERI, I want my software to be safe on unsafe platforms.
I am building a memory-safe language that I will rewrite my software in.
1. A rules-based approach to define a minimum standard, likely based around existing descriptions of software weaknesses where the rules basically say "you aren't allowed to have this type of software weakness" (the doc specifically gives path traversal as an example). HOWEVER, liability would only be established if a weakness is actually exploited, to help avoid the problem of people proactively scanning for weaknesses and then troll-suing as a way to make money.
2. A process-based safe harbor to avoid liability for exploitation of weaknesses NOT on the rules-based "no no list." Basically, if you can show you followed the safe harbor process requirements, you get the benefit of the safe harbor. This is to protect and encourage organizations to go above and beyond the floor established by the rules-based standard.
Agreed. Torts require damages.
Not to get too in the weeds, but breaches of implied warranties were once/began as torts.
But, yes, I guess I was a little glib. I should have said -- it would be sensible to require damages for these types of actions.
Although I'm not sure I agree with the author's formulation:
> HOWEVER, liability would only be established if a weakness is actually exploited, to help avoid the problem of people proactively scanning for weaknesses and then troll-suing as a way to make money.
I think we should encourage customers to look for vulnerabilities. Damages analyses can take into account depreciation, reasonableness of damages/troll suing, etc. If you have 100s of devices you must replace because the vendor won't patch, and your patch request is reasonable (high consequence vulnerability) within 5 years of purchase, I think damages accounting for depreciation make sense, without proof of exploitation.
EDIT: Implied 2nd quoted portion was the parent commenter's notion when it was a summary of the author's position.
I suppose you can construct a regime in which feds come in a fine you simply for having an insecure device, but a lighter touch approach is to wait for damages, which given this scenario, perhaps never comes.
They should pay to replace the functionality, and some punitive fine, as a start. Then we can get into whatever damage their negligently designed device did (did it participate in a DDOS? Did it attack other devices on your network?)
Ahh.
Simply having to replace a defective device would be "damages". The only Q would be: Was it reasonable to replace given the threat?
Nope, the recall usually just means you bring it to a mechanic and they get a replacement part that does basically the same thing, but hopefully correctly this time.
Fixes like yours… hey, if it works it works, right? But the device is not exactly what was sold, if it had an Ethernet port and presumably some corresponding network functionality, there’s some value they ought to owe you for. Maybe the fix could be to ship out locked down raspberry pi to sit between the device and the rest of the network, haha.
If software becomes a product does that mean it's no longer speech? How do you have liability for something that is copyright? What about the first amendment in the us?
What's the standard for liability for something like a manual on how to repair your own car brakes, where if you mess it up you might get killed? How about medical advice online, where if you follow it, it might kill you? How about a manual on how to prepare fugu? How about if I give what I think is good psychological advice, and they commit suicide?
If I write any of the above, and it's wrong, and someone dies because it was wrong, how liable am I under current law?
How about if I write something that someone takes as investment advice, and they follow it and lose a lot of money? How about if I write something that looks like legal advice, and they follow it and end up in jail?
What is the actual current state of the law on these things in the US?
https://en.wikipedia.org/wiki/United_States_free_speech_exce...
Intent matters in the case of the first amendment. Or you limit how you can license software, that's tort, but someone will argue that your limiting what I can do with my speech at that point in time...
This change looks like the "Nuclear option" for something that is, fairly common in society.
If you look at 3d printed gun plans and code, 1A has arguably helped with keeping it on the net but the authors are mostly under psuedonyms.
The better alternative would to be to completely rework the software developing discipline to be as rigorous as civil engineering from the top down. Requiring proper licensing, and security clearances for accessing U.S. customer data.
Edit: I suspect any reasonable law would account for that, and these are the folks who do the Lawfare podcast, right? It is pretty good. I haven’t finished the paper yet but I’d be surprised if they missed this.
That said, there's a state of tension on this topic right now. The European Union's draft Cyber Resilience Act has included language across multiple versions that would in at least some cases assign liability to producers of open source software. They've tried to modify the language to exclude non-commercial open source projects, but there's been a lot of wrangling over the exact definition of "commercial."
That's disturbing since the definition should be trivial!
If I publish a library (whether open source or not, doesn't really matter) and charge you money to use it in your product, that's commercial. I'm happy to take the liability (and will of course charge you enough money to make it worth it to me).
If I throw out some code on github but you're not paying me, that's obviously not commercial.
I’m quite glad they aren’t ignoring the complexity.
It used to be this way (at least in the companies I was involved in back then). In the 90s there was plenty of open source but it was generally a huge no-no to use it in commercial software (even when the license was permissible).
Incorporating an open source library (just one!) to our product in the 90s meant months and months of meeting with company lawyers to get that approved. And it meant we (the team developing the product) were 100% on the hook for all fixes.
While I'll admit it was a pain, in hindsight there was a lot of benefit from that approach. It discouraged importing random libraries unless there was a lot of value in them, so we had to be selective. It made it crystal clear that open source is not free, there's a lot of cost and liability.
While it's convenient today to import a library that brings in 700 other libraries, none of them vetted, any one of which might be full of malware, maybe that's actually not so smart.
I increasingly feel we need to go back to explicitly admitting that we (the company) are 100% responsible and liable for the code we ship. If we import it from open source, no matter, we're still on the hook.
Yep, and they had a podcast episode with the author of this paper: https://www.lawfaremedia.org/article/the-lawfare-podcast-jim...
My POV it's better than pretending certain segments of the economy are special and can completely avoid all liability.
People really hate the idea of slip and fall cases until their grandmother slips and falls.
> The better alternative would to be to completely rework the software developing discipline to be as rigorous as civil engineering from the top down. Requiring proper licensing, and security clearances for accessing U.S. customer data.
Yeah, maybe. It's kind of incredible this kind of seat of the pants engineering has lasted as long as it has.
Which is why it is a lawyers dream unless it is completely reworked.
I'm really not sure this is case. I'm sure there are plenty of people who would have argued medical practice is more of an art, than a science, before the introduction of strong med mal regimes. Everyone tends to think their field is special!
If we go down this path then we need proper licensing, certifications, and changes should be as well tested as the fda process. Aka releasing software is as rigorous as releasing a new drug.
P.S. where should the liability fall for:
This site can’t provide a secure connection www.lawfaremedia.org sent an invalid response.
But products liability is for products? If you aren't selling anything how would you be liable? If you and I have no business relationship, what is the basis of liability?
I don't see a liability regime touching internal use, personal use, or non-commercial (read: FOSS) development. However, when you decide to sell a service or a good, that's when liability attaches.
> This site can’t provide a secure connection www.lawfaremedia.org sent an invalid response.
Works for me?
Not in traditional tort law. A criminal stealing your car is an intervening event. You didn't cause the harm.
But for the sake of argument let's say you find a precedent. Please explain how that liability would work in the context of products liability.
First ask yourself the threshold question: What is a product? A product is not your FOSS on GitHub.
>"Please explain how that liability would work in the context of products liability.
First ask yourself the threshold question: What is a product? A product is not your FOSS on GitHub. "
I think that a court could easily determine a FOSS contributor was liable if they knew of a way in which the use of their software could cause harm to a user or 'victim' of the software. I am not sure the word 'product' does any work here.
> The majority common law rule among the 50 states is that the owner of a stolen vehicle will not be held liable for damages when the vehicle is stolen and then involved in an accident that causes injury or property damage.
This is a list of states which have varied from the majority common law rule!
If a state creates a statute which says you can be fined for selling ice cream on Sundays, guess what? If your fear is the feds, like these states, may do something crazy when writing the statute, that may be a legit fear. But that doesn't have anything to do with what are traditional common law tort principles, or with the content of the article.
> I think that a court could easily determine a FOSS contributor was liable if they knew of a way in which the use of their software could cause harm to a user or 'victim' of the software.
You think? I suppose you can think anything you wish. But you're going to need to show your work. Like -- here are the elements of a product liability tort. Here, is how the facts apply to those elements. Conclusion, liability could result.
> I am not sure the word 'product' does any work here.
It does all the work I need it to do? Your core problem is a "product" must be sold. Your/our FOSS is not sold unless you/we sell it. If some ridiculous goofball takes my FOSS and puts it in a product, and there is a problem, that's his problem because he made it a product and sold it, not mine.
Then that SaaS company would be liable (unless they purchased nginx)? If you take someone's FOSS and create a product with it, then of course your customers will look to you when that product has a defect.
- Why should one be held liable or have a legal obligation to the customers of the megacorp?
- Why would one even be afraid of being sued by the customers of the megacorp? (other than if we lived in an otherwise draconian society)
He wants a liability regime, and I have one. [2]
Yes, we need to start accepting liability for our software, and as he says, it's better to get one now before one is thrust upon us.
Bonus points if it funds FOSS.
[1]: https://gavinhoward.com/2022/10/we-must-professionalize-prog...
[2]: https://gavinhoward.com/2023/11/how-to-fund-foss-save-it-fro...