Move Over, Software Developers – In Cybersecurity, the Government Wants to Drive
techdirt.com
techdirt.com
The Administration will work with Congress and the private sector to develop legislation establishing liability for software products and services. Any such legislation should prevent manufacturers and software publishers with market power from fully disclaiming liability by contract, and establish higher standards of care for software in specific high-risk scenarios.Windows, back in the day, was both buggy and full of security holes. It was also $99, and enabled an inexpensive computer to do near-workstation feats. If it cost, say, three times as much, that would have changed the market dynamic quite a bit. Fewer of us would have been able to afford computers with Windows, and few of us wanted bare-metal computers.
Well, you say, that's just fine, we could have put Linux on it, and the world would have been a better place. Well, could you have put Linux on it? Can Linux survive in this regulatory environment? Can it survive as open as it is, as modifiable as it is, as free as it is? What about the rest of free software, that doesn't necessarily have a large company behind it?
The second-order effects of this could completely change the software landscape. We may not like the results.
Given that everything's getting either a WiFi or bluetooth chip in it, could we see coffee makers that are not approved for use in a commercial, health-care, industrial, or business setting?
You just can't certify your way to resilient software.
People are smart, they would have figured out really solid mechanisms that gave you great software that’s actually safe. Look at any of the fields that deserve to use the word engineering for examples.
Could high risk be the ability to monitor heart rate, or control motors? Could it be example code for those abilities that make them liable?
If RaspberryPi or Alphabet releases an educational board for experimenters but has features that could be used in a way that is considered high risk and if they are considered market powers, should they keep it from interested people because they can't fully disclaim liability?
How much should they reduce their feature set so that it doesn't tangentially run into this law. Can they just disable that part of the code, or would they run into the same problem as the GTA Hot Coffee case if it isn't gutted enough.
While it has good intentions, it will likely backfire some way as the lists of prohibited things grow or the market power clauses are reinterpreted to be applied to a wider range and smaller targets.
It's interesting that they use both the phrases "manufacturers" and "software publishers".
Obviously Microsoft, Oracle, etc are both manufacturers and software publishers but what about that random open source project? At minimum, they're "software publishers".
The "with market power" clause might seem like a protection but it's trivially easy to see many major open source packages with huge market share as having "market power". In that case, it's less about revenue and more about adoption and interop.
Frankly, this is the nightmare scenario that I thought Microsoft, et al would use to kill Open Source ~20 years ago. If the Feds demand liability, that kills OSS both from a “no company” standpoint and from a licensing “as is” standpoint.
Even if they don't outright BAN it, they could just make it painful to get approved, which is nearly the same result but lets everyone keep their hands clean.
Concrete example: I wrote a cryptographic library, which I give away for free. No guarantee, no liability, if I make a mistake the only thing you can take away is my reputation. But what if you want to use my library, and don’t want to be liable for it? There are two alternatives: you can approach me and pay me for me to accept liability… or you can approach a reputable audit company, and have them accept liability for a given version.
Legislate like that, and I’ll still be able to contribute potentially critical code for free.
And yes it is a risk. And yes I had one critical bug in 6 years, the horror. I did it anyway because it was pushing the Pareto envelope (speed/size mostly). And of course I knew what I was getting into, which is why it is not a weekend project, but a professional grade library. More explanations on my choices here: https://monocypher.org/why/
And by the way, Monocypher is currently being used to operate the TKey from Tillitis. https://tillitis.se/products/tkey/ Good luck porting libsodium there.
---
You do have a point however: actively distributed critical software with no one being liable is an anti-pattern. Though strictly speaking, my code is only critical when it starts being used, so… we’re going back to the place of amateur work: if you want to use my code, you can pay me to get a license that does not waive my liability, or you can pay someone else to audit my library, or you can be liable yourself. And if I sell my code, I should indeed be automatically liable.
The point with my example is that with proper legislation, even cryptographic libraries can continue being open source.
“Professional grade library” is an interesting choice of words in a thread about how we need to have liability for software, especially for software that is today in use exclusively licensed under no-liability licensing. If anything, all of the words on your site make it worse. A reasonable person, upon reading those fancy words, would be lead to the conclusion that this is Serious Software and thus safe to use.
Are you liable under the license they’re using? Their site has a lot of “trust me bro” security words on it, which is a great illustration of my point. You wrote it, you chose of your own free will to make it available, you’re responsible if it gets people hurt.
> The people who wrote OpenSSL thought they knew what they were doing.
The people who wrote OpenSSL had a much harder job, and my epistemic position is a lot better: I have over 20 years worth of additional collective wisdom to draw from, the primitives I implement are much* easier to deal with, and the problem I set out to solve is at least one order of magnitude easier.
> Are you liable under the license they’re using?
I’m not. And to be honest I’m on the fence on whether I should be. Nobody paid me for this work after all. On the other hand, if a law made me liable I wouldn’t complain too much to be honest.
If it’s regulation more than liability, I already followed most, if not all, the procedures outlined in the upcoming EU law on software resilience. I should have little problem going all the way to full compliance there (except perhaps if they ask for third party audits, I’m not paying 6.000€ per release).
> you’re responsible if it gets people hurt.
Believe me, I know. Thankfully so far nobody reported being hurt in any way — even from the sevcrit. And I continue making sure it stays that way.
---
By the way, I’m planning to work on an FPGA RISC-V core soon. I’m pretty sure nobody will give me any flak for this. And yet when you look at it, this is even more critical than a cryptographic library.
You seem to have totally missed the point of this thread. That’s not the best you can do, it’s merely the best you’re willing to do. You’re so convinced that the internet should trust your code? Put that into the license. You have the directionality wrong; nobody asked you to release dangerous code while disclaiming all liability. You decided to do that.
OpenSSL or log4j would like a word about how shallow bugs are in popular software.
The "private sector" here will be FAANG and other huge tech cos. They will argue for massive liability. Congress will agree because it makes them look tough and decisive. They know that only they can afford such liability. Smaller companies will be crushed by the liability insurance they need to cover these requirements.
On the other hand, as a rando off the street consumer, I appreciate that an FDA exists to help make sure someone selling me bread isn’t mixing sawdust in the flour to help juice those sweet startup bakery numbers. There are plenty of places to criticize the FDA, but we wouldn’t need that regulatory oversight if the public weren’t being harmed by such practices.
We should be perfectly happy to trade in higher quality but dangerous products for lower quality but safe ones. The airline industry kills many fewer people than it used to, and most crashes and fatalities happen in the less regulated parts. Because of course they do.
tiny, very tiny, particles of saw dust (labeled "cellulose" in the ingredients), is used to keep the cheese from clumping up.
go and watch your favorite pizza chain make pizza, with giant bags of pre-shredded cheese, and then ponder your sawdust intake.
Edit: while we’re playing food trivia though, ponder the no-sawdust rule that doesn’t forbid cinnamon.
At the time, it was common to put formaldehyde in meat and dairy, borax in butter, salicylic acid into wine, and to sell products such as “pepper dust”, which was literal dust with no real pepper in it.
I would have a lot more faith in regulation as a spur for security innovation if the overwhelming proportion of self labeled security people weren’t just obstructionist grifters.
And I would have a lot more faith in software developers if they didn’t go all in with npm madness and like to tell anyone that points this out that they’re out of touch.
Or when we collectively take a decentralized system and use it in a centralized way because we are too lazy.
One of the clearest things we must get a handle on is auditing where code came from, and on which machines, under which jurisdictions, any building, packaging, signing etc. occurred.
I would tend to also believe we need to have much stronger code permissions enforcement, especially moving towards a whitelist only model around network host access. If we can enable only sub sections of our code to have such permissions so much the better.
What we must not do is allow a few government supported gatekeeping organizations to act as man in the middle signatories, where dealing with them is mandatory to deploying anything. That is a government run App Store.
In most industries that I worked in, management folks do not understand how graciously they have fucked up for politicians to care about something unmeasurable as cyber security.
A lot of developers I have spoken with think that workflow patterns like Test Driven or Domain Driven Design will make their code inherently more secure. Even when you point out absolute rookie mistakes like missing sanitization of user input they're always smart-assing you about how this will never affect anything that they built. If you then point out RCEs in libraries they keep reintroducing for whatever opinionated reasons they deflect the blame upon the random guy on the internet that built this library on the weekend.
I am glad the US made the first move after the pipeline incident, and I am glad that government projects started with SBOMs. Why? Because it showed up how little companies know about their risk management of operational dependencies.
It's easy to blame the log4j maintainer, but it isn't easy to blame the guy for a mistake that he made more than 20 years ago, and fixed it meanwhile while already documenting it. Because that is totally upon software/development managers and the total lack of quality assurances and testing in their workflows.
And that is talking only about something as easily fixable as a java dependency in your xml file.
Imagine how messed up security could be if there's a babel library affected, and joyent would not hard-fork it to force-fix it for everyone. There's no way to guarantee anything in npm, by nature of the pre/post hooks in its package manifest format.
But this is exactly the type of mistake that government bureaucrats always make. They focus completely on procedure. There will be a whole bunch of boxes to check, including "this project uses TDD," "this project uses Domain Driven Design." Check them, and you are declared "secure." This is why such regulation will fail to provide any security, and will just slow down development.
And the max government salary is around $170K, no bonus, no equity. It's not like they're going to be able to hire any competent engineers to write these rules. They even had to build the US Digital Service, which is basically a charity case (for the richest most powerful org on earth, lol) to even get their house partially in order.
I don't think this will work in LLVM based compilers (due to the nature of how a backend compiler is being implemented for each platform and architecture), but maybe we should take a closer look at ADA Spark, or maybe we should try to focus our efforts on satisfiability resolvers that can point out common mistakes?
Golang kind of made the right first step with their vulnerability checker that is AST aware and can point out methods and signatures that are known to be affected.
There's also a lot of complexities behind decompilation aspects so I don't think C ABIs will be enough in the future, and a more data type aware format is probably needed.
I then mentioned that in their current software, all the URLs were directly accessible, since the image requests were not checked by their terrible, terrible, hand-made, bespoke, authentication and authorization system. They said basically that was okay, because the URLs were complicated. They weren't, it was a path and a sequential image id. They wanted to pay for features but not to fix an obvious security hole. And that's probably the norm for most customers - why do they want to spend 300k fixing bad, old code that they think works?
Most developers do care and try to make correct software. Some developers are ass-hats that are barely competent, going from project to project before they are 'found out.' Some contractors will put anyone with a pulse on the contract to avoid losing billable hours. And when's the last time you heard someone putting off new features if their competitors were already rolling similar features out?
That being said, I can see this resulting in the market for at least some products completely drying up, or getting very expensive. It would basically cement some players into the market, like IBM/RedHat for Linux, ARM and their toolchains for embedded devices, or Microsoft/AWS for cloud. It would be a way to create a moat against competition. It will probably spawn and insurance market, making it impossible for small developers to also purchase the policy to release their software. Maybe you could add additional clauses to open source licenses, such as 'not licensed for use in any sensitive or critical system as defined by USC x.y.z'.
And it still doesn't fix the problem of the pointy-haired boss ignoring risks or releasing too early to meet a schedule or competitive pressure.
Have you considered hidden watermarks just like printers? :)
Real example, I was working on TPM provisioning once. There was something fishy about the process: though they ostensibly wanted a secure channel between the provider and the TPM, the connection procedure didn’t properly authenticate the TPM. It could be replaced by a fake or emulated TPM, the provider would be none the wiser. Not too big a deal (it all happens inside the factory), but still worth pointing out. Anyway, my team lead did not believe me.
Fast forward a number of weeks (the TPM is a freaking mess, I was slow), most of the provisioning was done, secure channel and all, the whole procedure worked without a hitch… on an emulated TPM. Told you so: we need to validate the public key shown in the certificate the TPM provides, else it could come from any manufacturer. Anyway, I had working code to prove my case, and my team lead still did not believe me.
Me, the guy who wrote a cryptographic library, and was put to work on the TPM precisely because of that expertise. I had to escalate to the security guy on a meeting, and he believe me. Probably because he actually knew his shit. So at last, I was allowed to fix the bug.
The lesson I draw from that is that demonstrating a security exploit is freaking important. It’s not enough to point out the bridge isn’t finished, people need to see a car falling down to its doom.
Its concerns around limiting free expression, duty to adhere to standards of care, and market power could be alleviated by suitable legal tests.
I agree liability for vulns in court is problematic because of the elaborate possible futures that can occur in court, and the costs involved, but I see that this would likely be set by a sensible implementation of these ideas as a "last resort", with plenty of escape hatches to avoid it on the way, leaving the effect not toothless but just.
I'm interested to see where this will go, I don't think it will be a bad thing for my company's cybersecurity products.
A potential for abuse tho I think is the misuse of whatever collection of laws and rules results, to compel and gag deliberate weakening for the purposes of maintaining backdoor access, which should instead be negotiated through dedicated channels (warrants etc).
Whenever there's commentary on new laws, it's pertinent to ask what lobbyists might be representing what interests in any discussion? I don't know the answer here, but I'd like to see speculation.
One awesome thing that I hope results from this is that more lay people get somewhat conversant in what constitutes secure by design software. An increase in such security literacy can only be good--I think! Unless it's like that situation where people go to their doctors with their diagnosis and treatment already worked out thanks to Google, and are just looking for the doc to rubber stamp it! Which...depending on you and your doc, might be the right choice! Haha :)
This will never happen anymore than laypeople will start thinking about the engineering principles behind building the reliable internal combustion engine in our cars. We want things that work without us thinking about it, and that's the end of it.
In other words, I think there's a certain level of trickle down expertise, and in fact, verily that's how culture grows its knowledge over time. Look at the history of hygeine: at first people laughed in the face of invisible germs, now people know to wash hands, and even, recently, they wear masks when sick. Among a plethora of other hygenic practices. Yes, there will always be hold outs: dubious food-truck kitchens I be lookin' at ya--but overall I think people do learnings over time and pass em down to the subsequent generationisms. Or don't ya think so? :) ;p xx
But.
Isn’t it past time we actually get accountable for our work? Isn’t it past time we provide some form of guarantee of merchantability and suitability for some purpose? Sure amateur work should not be forbidden. But come on, when we sell something for profit, we should be accountable when it falls short of its promises.
I don’t know how to best handle it. I’m even pretty sure some unintended consequences are inevitable (though they can probably be minimised). Still, do we really want to keep the status quo, where we can sell insecure software and never be liable for this? Most other industries are liable already, why wouldn’t we?
One thing’s for sure: voting with our wallets clearly isn’t enough. One way or another our market needs a little reshaping.
Single player game: maybe we want to cut back on the skinner boxes, or automatically mark 18+ any actual gambling.
Professional accounting: maybe we want the thing to be error-free under normal operating conditions (good luck defining "normal" here though).
Media player: that thing routinely reads stuff from the Internet, maybe it should be robust to hostile input.
This is true in almost every industry, especially medicine. There is no accountability.
It's a myth like the fairy godmother, etc.
And which if you audit all those packages, 10 seconds later they could be sold/stolen/hijacked for malicious intent...
Yet devs think this is awesome...
Shows why we're stuck gping dpwn this path.
Because devs have shown, they do not care about security, at all.
Could add a grain of salt there, but I get your sentiment and would not want to argue their are big problems in the software industry, supply chain being one.
We could be angry about that, but what can be done to improve the situation? Specifically: what should the government do?
Proposed solution of adding a safe harbor framework to shield [companies] from liability is like saying: don't use npm, use our blessed supply chain. Sounds like a bold experiment, but IMO it's not obvious that this will curtail the reckless behaviours of those software devs you refer to.
Is the blessing done by you, someone in your team, someone in another team, someone in another company, someone in government?
Is the blessed supply chain comparable (not in volume, but in features, ease of use) to NPM or pip ecosystem? What do you do when you need the feature that isn't in the blessed set?
A lot of the scanning is automated and takes care of problems I've actually run into, like virus laden files. if something is not available, we don't use it. Because the cost of violating that policy is worse than having to find another solution.
Like most devs (or engineering managers), I’m resigned to modern realities.
It’s a race to the bottom and I’ve been dragged into flame wars by smug (and very wrong) engineers who believed semantic versioning would solve compatibility problems and the like.
It’s impossible to justify the cost of full auditing and ground-up building. The only way to solve this is to force ground rules.
Regulation can’t come to this snake oil industry fast enough.
Besides, I’m of the opinion that knowing your dependencies is a requirement for all serious software. (Meaning, if you’re not already tracking your dependencies, you are not taking your own software seriously.)
Maybe other platforms don’t have the same scale of horribleness as the web stacks, but there’s still a problem.
For the dotnet ecosystem, for instance, I’m not reinventing functionality when nuget packages exist.
But the same concerns persist because I’m not really sure if it’s secure, without auditing.
Both from a security and compatibility perspective, it's not a solved problem because it's not a problem specific to any tech stack.
1. The most obvious direction is that everything slows down in development. Innovation is killed in established businesses, startups need more initial investment.
2. If startups are not affected like in the EU, then there will be an explosion of them as large companies outsource their innovation or spin off their own startups. Innovation within large companies will still suffer. Maybe that's a good thing.
3. Greater focus is put into fewer projects to develop a stronger ecosystem and engineering culture so that we can increase productivity and still have time for strong standards. Might end up like 1 anyways.
But I doubt the EU would be any more lax over time.
And it could spawn a boom for compliance consultancies that can maneuver the regulations and check the boxes on behalf of companies so that they can remain free from liability. All without any meaningful impact.
I would love to believe we could inch a little toward:
> 3. Greater focus is put into fewer projects to develop a stronger ecosystem and engineering culture [...]
A hopeful sign might be stuff like PCI compliance: major impact on velocity, money wasted on consultants, brain-dead training and project managers, but also much less catastrophes than what might have been. But it's just as likely it goes the SLA-way, where promises and legal documents paint some desirable level of control that in reality does not exist, just because asses need covering.
Large companies will love this because now they can pay their engineers with options on the "startup" which will most likely fail rather than RSUs on the main company which are as good as cash.
And if controls fail and a Bank buckles down, they need to respond with their own capital. (Political favors and bailouts aside)
And what's considered 'critical' might be overly broad. Is the manufacture of Kleenex brand facial tissues critical? It might be under the final law. If your SaaS is used by Kleenex, do you now have to produce design documents, software process documents, audits of your process, etc. to show you can continue to be used by Kleenex to produce a trusted name brand in facial tissues? Do you do all the work to keep Kleenex as a customer only to have them switch to your Indian competitor who does none of that?
Inquiring minds want to know.
How many other world changing technologies were allowed to become so utterly fundamental to the functioning of society before we started making rules to set a minimum standard?
1. Invent new technology that revolutionizes society (e.g. railroad, telephone, software) 2. Quickly amass a fortune exploiting this new technology. 3. Use a portion of that fortune to lobby for industry regulation under the guise of concern for customers which just so happens to also raise the ladder and make it harder for startups to disrupt the incumbents, thus securing their fortune.
That said, even with the cynical take, I find it hard not to be in favor of regulation (assuming it's done well, which has been a bit hit and miss over the years). A lot of software has potentially severe liabilities and it's time companies took responsibility for those instead of shifting them onto customers.
Control would make sense if governments had a clear, flawless track record of developing software; that state of perfection could be attributed (solely) to a bunch of existing government guidelines and the only problem would be that developers fail to meet those guidelines. We're not there.
Why? The government does not have a clear, flawless track record developing food, nor medicine, still it controls and regulates them for the safety of society.
You can't demand absolute knowledge as a pre-requisite of regulations, that's absurd and unachievable. Extremely black-and-white thinking as well.
What I meant to say is that as it stands most governments aren't a shining example when it comes to overseeing their own software efforts.
Imagine a parallel world where where we no fuses, nor circuit breakers, nor any type of thing. You'd have government very carefully regulating what could and couldn't be hooked to the local power grid, just like this.
We live in a parallel universe to that, where nobody even considers only giving a program or "app" access to a carefully contained set of resources, instead of the whole computer, and internet. We find "as if" solutions, like VMs or containers, while still ignoring the root issue.
Wake up!
Now, they start to tell the private sector how to do things. LOL.