Mitigating Memory Safety Issues in Open Source Software
security.googleblog.com
security.googleblog.com
After some clicking around, I was directed to the OpenSSF mailing list [2].
…where my request was left to rot, without any response.
Made me wonder whether this is just a PR stunt / corporate power grab.
[0] https://opensource.googleblog.com/2020/12/finding-critical-o...
[1] https://news.ycombinator.com/item?id=25381397
[2] https://groups.google.com/g/wg-securing-critical-projects
Hey, I am involved in the OpenSSF and happy to take at least a look. I maintain a popular python OSS project in the security area so we have that in common. Did your email actually go through, I just searched my list archives and there is nothing for gensim. EDIT: see your email now, have msg'ed the WG in slack.
> Made me wonder whether this is just a PR stunt / corporate power grab
I would not say it is, we have folks from OWASP on the technical advisory panel and many other non profit orgs / individual developers. One thing I would keep in mind, is that being a newly founded org, a lot of time has been spent working out and agreeing how we will operate, so the working groups are all still quite fresh.
https://www.googleapis.com/download/storage/v1/b/ossf-critic...
In the list of all projects, it is 2909th.
https://www.googleapis.com/download/storage/v1/b/ossf-critic...
You posted in December that it was #119 for Python. That suggests to me that this list might be quite volatile, which calls into question its usefulness.
Regardless of the actual position in the list, your email should have got a response!
Correct me if I’m wrong, but that’s my understanding of the rather terse article.
I would rather have the original maintainers funded and still in control of the many pieces that form the basis of the major Linux distributions.
The ISRG seems to have two projects of this type mentioned on its website: an improvement to curl (https://www.abetterinternet.org/post/memory-safe-curl/), in which the ISRG funded the curl author directly; and an improvement to httpd (https://www.abetterinternet.org/post/memory-safe-tls-apache/), in which Google (via the ISRG) are funding an httpd committer.
> What is left out of the title is that they do not provide funding to the current authors/maintainers of these open source projects. Instead, they fund an organization that will rewrite these tools.
This's not what's happening. Here's what is:
ISRG plans and coordinates investments in moving open source software to memory safe languages. We have a strong preference for working on these plans with maintainers and developers, and then funding them to execute the plan.
Once we have a plan prepared, and, ideally, maintainers/developers on board, then we pitch it for funding. Once someone (e.g. Google) funds the project, then ISRG handles contracts with the project developers/maintainers to get the work done. In some cases maintainers may be on board but want us to find a contractor to actually do the work.
In both of the first two projects, and in most future cases, the money that Google (or another company) provides will largely go to project maintainers / developers.
Also, thanks for Let’s Encrypt. I didn’t make the connection between the two names. This initiative looks much better to me now.
Is rust the only language that you are considering? Would go make sense as well?
given the potential for security theater and general fud and the potency that appeals to authorities such as sponsors like Google can create in the minds of the public, how has this been evaluated in terms of the possibility for widespread disintermediation of the open source software community at large?
Incrementally – that means in-place. You can't fork and rewrite incrementally since there will be no adoption/users of the fork.
Also they talk about maintainers of the projects. I think that largely means original authors and maintainers, not forks.
I don't see how you come to that conclusion. Could you clarify? Thanks :)
Please don't make statements like that without doing a minimal amount of effort to verify.
[1] https://daniel.haxx.se/blog/2020/10/09/rust-in-curl-with-hyp...
And from link #2 that you posted, I see they talk about the curl stuff, but also show a bunch of repos with tools rewritten from scratch, e.g. https://github.com/ctz/rustls
So I really cannot draw a conclusion one way or the other.
Normally I’m an optimist, but after the recent chromium misunderstandings, I am going to err on the cynical side.
> Memory safety vulnerabilities represent one of the biggest threats to Internet security. As such, we at ISRG are interested in finding ways to make the most heavily relied-upon software on the Internet memory safe. Today we’re excited to announce that we’re working with Daniel Stenberg, author of ubiquitous curl software, and WolfSSL, to make critical parts of the curl codebase memory safe.
> ISRG is funding Daniel to work on adding support for Hyper as an HTTP back-end for curl. Hyper is a fast and safe HTTP implementation written in Rust.
I too found the post terse and the lack of quantification of the financial commitment makes me suspicious but I think they deserve the benefit of the doubt for now.
It'll be interesting to watch how this plays out. But I pity the projects where Google's gaze falls upon.
but if they rewrite the project in rust, why _shouldn't_ they control that project? If other people choose to switch, then it's not wrong for them to move, and not wrong for to google gain control of those users.
The original project maintainers is a third party to this whole process, and i would argue, is uninvolved tbh. It is up to the users of the project to decide to trust google's rewrite (and their tendencies to abandon projects...), vs the original maintainer's version, balanced against the security aspect.
If the rewritten project gets more successful than the original (perhaps due to corporate promotion), you have morally stolen the work of the original authors.
If you write a project from scratch without looking, this of course does not apply. But I doubt that is how Rust rewrites actually happen, the temptation is too great.
Ownership is by "moral" definition something completely arbitrary and made up.
If you work in open-source you most likely have different views on ownership than others.
By this logic, WINE is a derivative of Windows and should retain their license.
> but if they rewrite the project in rust, why _shouldn't_ they control that project? If other people choose to switch, then it's not wrong for them to move, and not wrong for to google gain control of those users.
I think the issue is the more critical projects Google controls, the more they can make decisions by fiat. Those decisions will favor Google's interests, and could sacrifice everyone else's interests in some way.
It's sort of like web browsers. My understanding is that Chrome is so dominant that Google can basically dictate HTML standards at this point, or at least effectively push them in directions that are favorable to its ad-based business.
> The ISRG's approach of working directly with maintainers to support rewriting tools and libraries incrementally falls directly in line with our perspective here at Google.
And a link to: https://www.abetterinternet.org/post/memory-safe-curl/
But if you're not the one doing the rewrite, why would google pay you? And if you would be capable of such a rewrite, may be they would pay you to do so.
And there will be some pro version that does not grab that stats, but you need to purchase subscription.
Sadly everyone expressing concerns like this will be dismissed and everyone is going to once again hail google for being community friendly and supportive of FOSS.
As others pointed out, it’s not clear who will do the rewrites. Based on the curl example, it might be the maintainers. It would have been great if they could state that concretely.
I’m also wondering why there’s a middle man instead of funding the projects directly, but that’s another story.
We have a memory safety initiative in which we plan and fund projects to move popular open source software to memory safe languages. For each plan we make we seek a funder - in a couple of cases now that has been Google, but it may be others in the future.
We are in the "middle" of the financing in the sense that funding passes through us, but the memory safety initiative is coordinated primarily by us. We select projects and work with maintainers to make plans. Part of that coordination we perform is matching up projects with funders like Google, and then managing the contracts and overseeing the work to completion.
If I understand correctly, this is an initiative that might have multiple sponsors, so it makes sense to have a central entity organizing it, and following up with all the projects involved. This makes perfect sense to me now.
Also, I was not aware that your org is behind Let’s Encrypt. This puts things into a very positive light.
It might make sense to post a sibling blog post/announcement to explain what this project is about and how it would work. I feel that Google’s post left a lot of questions to be answered.
If you don’t mind me asking, is rust the only language that you have in mind? Would go work as well?
There's nothing about this effort being focused on rewriting things in Rust.
I think you might see some full-system rewrites for serverside components where Go will work out. But I think the projects we're talking about, like I said, have ended up in this position mostly because they can't easily be written in Go, or they already would have been (if only be a competitor).
My guess is what will happen in the cases where Rust is unworkable is that there will be less-secure forks for the obscure platforms.
F-Secure has been using TamaGo for bare metal on their secure USB keys.
MeadowLabs is using .NET with a tiny C++ hardware glue for IoT.
Aicas and PTC are still in business delivering real time bare metal Java stacks.
Astrobe is still in business selling Oberon compilers for embedded development.
MikroElektronika is still in business selling Pascal and Basic compilers for embedded development.
And then there are those 5 surviving Ada compiler vendors, one of them open source.
Finally, if there is no way around having to use C, there is no reason not to use C++ instead other that anti-C++ hater religion.
Unless we are talking about UNIX clones, thanks to them C will still be around for several decades to come anyway.
That sounds snarky, I know. I promise, I'm not being snarky, and, again: it ain't my first choice (it'll be fine though).
As for the other platforms, and what Google is paying for, lets see where the bribe drives people into, after all from your point of view Google should rewrite gVisor and everything k8s as well.
Yes it definitely will, but not only, specially among groups where not using a GC is more of a religious issue than an actual technical impediment.
Windows is a good example of this politics game actually.
Joe Duffy mentioned on his Rustconf keynote that even with the system running in production, the Windows team couldn't be argued against.
A system that even powered parts of Asian Bing infrastructure.
Likewise they clearly sabotaged the Longhorn efforts, to replace them with COM, and yet UWP, mostly written in C++, happens to usually run slower than traditional .NET applications on Win32, or any other compiled GC'd language on Windows for that matter.
> That’s why we’re expanding our collaboration with the Internet Security Research Group to support the reimplementation of critical open-source software in memory-safe languages.
Though the rest of the article doesn't go into detail about this effort beyond http/TLS libs for curl.
Since most unsafe code is C or C++, Rust usually makes the most sense for rewriting one component at a time. It integrates with C and C++ very cleanly because it has good FFI and no runtime.
If we were rewriting more projects from scratch and integration with existing C and C++ code was not important, we might use other languages more often. I don't expect we'll attempt to rewrite many pieces of software from scratch.
Source: I run the memory safety initiative at ISRG.
Edit: the submission title is fixed.
There is a python library that recently switched to rust (partially) and now cannot be build on several platforms.
This also means that very long-lived platforms will probably be forced to upgrade that library every half year or so in line with the development of rust.
I'd say such an initiative needs a long-term supported version of rust, similar to C99 or so.
And this does not even account for cargo. I know you don't have to use cargo, but is that practical?
How are we going to maintain stable software for, say, ten years or more? Do we have anyone committed to providing a stable rust environment for these cases? Especially considering that rust has no ABI for dynamic libraries.
Oh and by the way: We need a way to bootstrap rust as well. The current distribution downloads binaries of the last one to build itself. And it ships with a vendored version of llvm.
So in conclusion: I really like the rust language. But I am concerned that it is still way too immature and to "hip" for long-running, stable projects.
there comes a point where we can't let dependencies on Amiga or DEC Alpha (companies/platforms that haven't existed in literal decades) hold the rest of us back. If someone wants to port rust/llvm to those platforms more power to them, but the rest of us don't need to suffer. The only platform that actually matters that was negatively effected by that change is musl/alpine because there aren't musl wheels available and you have to build the rust part from source.
What about embedded platforms, which tend to remain relevant (i.e. present in large numbers out in the real world) much longer than desktop or server platforms? Don't they matter? Don't they need updates, including for security, as much as any other?
As a security library... you really ought to be updating it every half year of so.
This has nothing to do with rust though. Rust maintains full backwards compatibility, so even if you upgrade the compiler (which is unlikely on long lived platforms) and not the library nothing breaks.
Nor is it necessarily the case that upgrading the library will require updating rust (though this isn't what you appear to be complaining about). Not only is it a build time dependency (that you don't even need to install if you are on common platforms which ave prebuilt binaries), but they are explicitly testing against a particular fixed old version of rust as well as recent ones: https://github.com/pyca/cryptography/blob/main/.github/workf...
> I'd say such an initiative needs a long-term supported version of rust, similar to C99 or so.
This describes every version of rust since 1.0. Rust is fully backwards compatible, unlike new C standards.
> Do we have anyone committed to providing a stable rust environment for these cases?
Yes, the entire rust team...
> Especially considering that rust has no ABI for dynamic libraries.
This is... rather irrelevant? Technically it does have support for the C abi for dynamic libraries though.
The fact that rustc is backwards compatible is great, sure. But it does not solve the problem of distributors.
Say you have libfoo-1.0 that uses rust and cargo 1.2 . A distributor that now releases their linux version and wants to ship libfoo-1.0 must consider the requirements for supporting it. This support usually outlives the upstream support.
If libfoo-1.0 uses C, for instance, it is highly likely that a distributor can backport any security-relevant changes from libfoo-1.1, 1.2, and later into 1.0.
In the case of rust there are two uncertainties creeping in: One is the highly dynamic nature of cargo dependencies. libfoo might depend on a myriad of crates transitively and rust simply is too young to provide any experience in how long they will be available or supported. And no, it is generally not a good idea to update to new feature releases for security updates. This is something that stable distributions explicitly try to avoid.
So our distributor now must consider if changes in libfoo-1.2 pulled in changes via cargo that cannot be easily backported because they are feature releases or have a dependency on a newer version of rust.
The second problem is the compiler itself. Any change in libfoo-1.2 might require a newer version of rust. It is great that they want to be backwards compatible but now our distributor must consider upgrading a substantial piece of software during the support period (and testing the backwards compatibility, of course) just to backport a security fix.
Please don't get me wrong, I really like rust and I think these problems can be solved. But I also like stable software and all these languages with their package repositories seem to simply ignore everything that's older than one or two years. If rust wants to become a new C, which I would very much like, it needs to demonstrate a certain stability.
> One is the highly dynamic nature of cargo dependencies. libfoo might depend on a myriad of crates transitively and rust simply is too young to provide any experience in how long they will be available or supported. And no, it is generally not a good idea to update to new feature releases for security updates. This is something that stable distributions explicitly try to avoid.
crates.io is designed so that dependencies put up on it remain available forever, and that the only other cargo related dependencies also have to be hosted on crates.io - solving the availability problem.
Of course if you're trying to backport a security fix to an old version of a transitive dependency that (like the non-transitive dependency) is no longer supported, you do have to make that backport. Once you make that backport it's easy to update the version of the transitive dependency with a [patch] section in the top level Cargo.toml.
> The second problem is the compiler itself. Any change in libfoo-1.2 might require a newer version of rust.
This is already mitigated in multiple ways. As discussed before, people do make sure to support old versions of compilers, you don't need to have a compiler on your target platform anyways. But moreover, the amount new language features make backporting difficult is dwarfed by the amount that internal library changes do.
While in C/C++ I can just build a fork as dynamic library and put it to the system image or to the own package manager repo.
I mean for sure Cargo is great for small hobby projects, but for the big complicated enterprise projects it's not really suitable. Even Google doesn't use Cargo in Fuchsia and vendors everything.
If you're making a small change, you use a [patch] section in the Cargo.toml for your workspace if you're using one, or the Cargo.toml for your binary/.so/.a if you're not, and it will cause your entire dependency tree to use your forked library. You don't need to fork the entire ecosystem as you suggest.
The [patch] works at the "workspace" level (and you generally should stick your entire complex system under one workspace), so if you use that feature properly for your "complex system" (keeping everything under one workspace), it is a one line change to replace the dependency for your entire system. If you use multiple, well, one per workspace that produces a binary/.so/.a.
Also, for big enterprise projects, I would assume you would use a private registry in the first place, which is apparently well supported (though I haven't used it). That's how the enterprise places I work at have dealt with python dependencies for instance (I haven't used rust in a enterprise).
I can sort of understand people trying to maintain LTS linux distros complaining that for their purposes this is a bit of a pain compared to C style shared libraries (because they need to modify a lot of rust workspaces)... but to be honest I think that's a question of their package management tooling not rust's. Rust provided the tools they need to make their custom tooling work. As a user the number of times shared libraries have caused me pain makes me think that this static linking is the superior method of packaging.
Also although we are talking about open source here, in many OSes, and enterprises, distribution of binary libraries is a very important subject, which cargo doesn't allow for.
In theory perhaps. But in theory this could happen with C or C++ too. In practice, security patches try to be minimal.
Sure, if you're several versions behind you might have more work to do. But you knew you were gonna own that problem when you decided to lag behind.
I could have got around it, but it took me by surprise that a Python dep required it. Not being negative or criticising the fact, just first time I had observed it.
Progress should not be held back for the rest of us because a few will get left behind. If you are running something so old that LLVM doesn't support it, upgrade. It's 2021.
ISRG plans and coordinates investments in moving open source software to memory safe languages. We have a strong preference for working on these plans with maintainers and developers, and then funding them to execute the plan.
Once we have a plan prepared, and, ideally, maintainers/developers on board, then we pitch it for funding. Once someone (e.g. Google) funds the project, then ISRG handles contracts with the project developers/maintainers to get the work done. In some cases maintainers may be on board but want us to find a contractor to actually do the work.
In both of the first two projects, and in most future cases, the money that Google (or another company) provides will largely go to project maintainers / developers.
> That’s why we’re expanding our collaboration with the Internet Security Research Group to support the reimplementation of critical open-source software in memory-safe languages.
Go / Zig / Pony / Haskell are all valid contenders.
Rust, Zig, and to a certain extent D or Nim without GC, can easily expose a C API and have a minimal runtime.
But the others do not provide the same guarantees as (safe) Rust with the borrow checker. All very cool languages in their own right, but memory safety in a no-GC environment is not one of their defining attributes.
Note: a dependency on any of those compilers is still far from trivial for many projects.
As the ceator of D, how do you see the place of borrow checking in the ecosystem long term?
Would you imagine most D code to be written against the borrow checker in the future, with transparent convenience for GC types?
Or would it remain a somewhat niche feature for domains that can benefit?
I don't know how pervasive its use will become, but I expect its use will steadily increase.
it's truly superior to any other programming language (5+ years of pro experience talking), so why waste my time with the manual labor of imperative programming?
> why waste my time with the manual labor of imperative programming
Someone has to. https://github.com/ghc/ghc/tree/master/rts
It's also introducing isolation [3] to provide safe multithreading similar to Rust's sendable.
[1] https://nim-lang.org/blog/2020/10/15/introduction-to-arc-orc...
> ORC is Nim’s all-new cycle collector based on ARC. It can be considered a full-blown GC since it includes a local tracing phase (contrary to most other tracing GCs which do global tracing). ORC is what you should use when working with Nim’s async because it contains cycles that need to be dealt with.
Sooooo unless you
> disable the GC and do manual memory management, but then you lose access to most of the stdlib
you still have to run a memory-managing runtime
Although I find it interesting that Common Lisp gets left off these lists. It's memory-safe (with GC of course), has great integration with existing C libraries via CFFI, has a formal spec, has a solid library ecosystem, supports a "mostly-functional-but-imperative-if-you-need-it" programming style, has optional/gradual compile-time type checking, and at least one open/libre implementation generates fast native code (SBCL).
Might as well throw OCaml in the mix too, since it has all of the above minus the formal spec.
I remember a recent HN comment said that if you create a website for your new programming language, you should always ensure there's a meaningful example right there on the landing page. A pity that Pony's page fails to do this. Here's one of their example programs. [1]
[1] https://github.com/ponylang/ponyc/blob/main/examples/timers/...
It seems like it's best grouped together with Go, Erlang, Elixir, and Crystal.
But I have always used Pascal for that. It has memory safe strings and arrays with refcounting, so long as one avoids using other features like pointers, classes, inline assembly, it is perfect.
Edit: here's the link, interesting idea. https://github.com/google/wuffs
The memory-safe languages can still panic or have integer overflows, Wuffs should prevent even that
* Languages with significant runtimes are going to be rejected by project maintainers; nobody is going to rube-goldberg a GC into an existing project just to replace a risky component when the option not to do that exists.
* Languages with insignificant adoption are going to be rejected by project maintainers because why take a flyer on an unknown quantity (not just technically but in terms of where the community will be 5 years from now) on a project you've been maintaining for decades? Say what you will about the Rust community and its long-term financial viability but the industry is pretty much already committed to it.
I'm fine with Rust but, like, not it's biggest fan (strongly prefer Go) but Rust pretty clearly hits the sweet spot being targeted here. I think you can assume safely that most of this work will be done in Rust.
There are a lot of tricks needed to make the above work, and details that you need to consult a lawyer and accountant for.
but if they develop it for internal usage only, then they don't get the added benefit of destroying the existing copylefted version!
I'm also curious if they will accept GPL or not
"Insecurity is freedom."
The "recent study" quoted in the article was published at the beginning of 2019, using older data.
Current vulnerability data shows that curl has very much limited the risk of memory safety issues. How many reported security vulnerabilities in the last two releases of curl? Zero so far. You have to go back 9 months before you find one expired-pointer derefence issue resulting in potential data being sent to the wrong connection in rare circumstances and configurations. Which is a logic error that could happen in Rust too.
To quote from a Oct 2020 study on Rust safety - "Our study results reveal that while Rust successfully limits the risks of memory-safety issues in the realm of unsafe code, it also introduces some side effects. In particular, most of the use-after-free and double-free bugs in our data set are related to the automatic drop scheme associated with the ownership-based memory management model." -- "Memory-Safety Challenge Considered Solved? An In-Depth Study with All Rust CVEs"
They study 60 Rust memory safety vulnerabilities.
As you can see not only does Rust not solve the memory safety problem, it has other issues. Additionally the old research quoted misleads people about the current status of reported memory safety issues in curl.
It also finds that Rust has novel patterns of unsafety in unsafe code. That's important! But it's fully compatible with the claim that Rust is much safer than C overall.
I don't think it ever says that the sum of safety in safe code and novel unsafety in unsafe code adds up to as much unsafety as C. The paper's overarching claims aren't quantitative.
I think you're overstating the claims of both the blog post and the study.
It also states that Rust completely prevents them - it does not. The article talks about mitigation, but also says: "would have been completely prevented by using a memory-safe language". The "completely prevented" claim in the article is the one not supported by current research. If you only read this article, you'd be mislead about memory safety in Rust.
Only if you look very recently. Earlier you said:
> You have to go back 9 months before you find one expired-pointer derefence issue resulting in potential data being sent to the wrong connection in rare circumstances and configurations. Which is a logic error that could happen in Rust too.
That bug is 6 months old and could not happen in safe Rust because references (pointers) cannot outlive their referents.
> It also states that Rust completely prevents them - it does not. The article talks about mitigation, but also says: "would have been completely prevented by using a memory-safe language".
It is literally true that they "would have been completely prevented by using a memory-safe language." The complication is that Rust is only memory-safe if you don't use unsafe.
rustls (the new curl component in question) uses no unsafe itself. I find some unsafe in its dependencies, but most of it seems to be for FFI, which is inherently unsafe. I'm also not sure that those should count—do OpenSSL vulnerabilities count as libcurl vulnerabilities?
> The "completely prevented" claim in the article is the one not supported by current research.
I will grant you that it's slightly misleading because it's possible to write unsafe Rust code. But that's not news, "current research" has nothing to do with it.
I'm assuming Go will also be an option, what are the others options for core OSS system tools?
No doubt, it will still be around in low latency or embedded. However, this is a significant cut of their market share, especially if Google, Microsoft, and Amazon retires its usage in its systems.
C++, on the other hand, has a variety of choices for build systems. CMake is prevalent, you can use simple makefiles just like C, or you could use countless other custom solutions. I’ve found that languages which support a singular build system are really easy to build simple demo apps, but once you need to build something that the default build system doesn’t support, you’re in big trouble.
I often renounce to contribute to C/C++ projects because I know I will spend more time figuring out how to install all of their dependencies than making my two-line fix.
On the contrary, in the rust ecosystem, I know I can just make my two-line fix, run cargo test, and if there is no red line in the output, I'm done.
> I often hear that it’s because of Cargo
I don't think the main reason is cargo. The main reason is that you don't have to worry that the very simple pointer dereference you just wrote may allow anyone on the internet to take full control of the machine of anyone that runs your software.
OTOH Cargo is the only option for Rust projects. There's nothing to bikeshed about. Everyone has it and knows how to use it.
Apart from that the project is about memory safety. Companies like Google, Microsoft, and Mozilla already have huge C++ codebases, top engineers with lots of C++ experience, spent tons of effort on making it safe, and still weren't happy with the results.
Maven != Cargo
> C++, on the other hand, has a variety of choices for build systems.
None of which even remotely approach the ease of use and maintainability of the Cargo build system.
> once you need to build something that the default build system doesn’t support, you’re in big trouble.
Cargo can build arbitrary dependencies using the build.rs build mechanism.
Indeeed, Maven supports binary libraries.
If you want a safer language with in my opinion a far better type system than anything Rust could possibly achieve at this point you can use Ada.
But why not those two options? Old, smelly and most importantly, difficult to deal with dependencies. You have to actually think about how your API will work in your library because if your library ever ends up used anywhere people are going to be running different versions and you don't want to ruin their day by having them change their code every time they update their machine.
In comes cargo (and friends, lots of other languages have picked up the same idea) where everything is managed by the programming language's choice of build system and API stability and dealing with dependency hell no longer exists when you statically link everything.
I think people don't care or want to care about why we got to where we are (distributions with dependencies and API stability and an aversion to micro-modules). A lot of people just want things to work when you run one command no matter how much complexity gets hidden under the carpet, how obscure the issues end up being and how windows-esque the debugging process ends up being when things do go wrong.
All this pushed by lots of big companies who don't care about the origins of the open source model and how distributions do things and instead only care about lining their own pockets.
Mark my words, distributions will disappear in a few year's time, people will cheer this on as some great technological advancement, things like snap will become the de-facto standard and debugging a complex problem will just never happen again. The internet will become filled with forum posts of how people "fixed" a problem by rebooting their machine or deleting a random file or re-installing.
Anyway, maybe I'm just a crazy person thinking up crazy scenarios and being irrationally scared of "progress".
It's not rust but at the same time I think rust has sacrificed general safety in favor of memory safety which has produced a tendency towards a false sense of security.
After all, it seems rust still has memory safety issues despite the borrow checker: https://arxiv.org/pdf/2003.03296.pdf
Yes, ideally the whole program would be rewritten but security is not about absolutes. For a manageable amount of effort they have reduced the attack surface significantly.
Honestly, I'm getting more and more disappointed by the quality of discussion here on HN.
There is no such thing as security critical code. If it is out there - it can be expoloited. There are a lot of stories with minor exploit chaining leading to huge compromise.
https://github.com/duneroadrunner/SaferCPlusPlus-AutoTransla...
Or encourage other memory safe-languages as well?
So what's the point of reimplementing one-off-tooling with under the same name (like sh?!) in Rust, except adding confusion and new bugs? And in a container, you already can run only your rust/Go-program, which effectively means that the main memory-unsafe attack-surface is actually the kernel (which they don't swap out either).
It would be nice if there was more meat on the article though, like what types of things to do, or how each memory error happens, etc.
I don't doubt that it will be a popular choice in that area, but even there it has competition from new languages.
And above all, I think it might take longer than a century to phase out all the C/C++ legacy code!
I could absolutely see C++ devs jumping ship to Rust. I think const generics just stabilized or is about to. That was one of the first shortcomings I noticed when I learned Rust. I think custom allocators is pretty much the next big piece that would give C++ devs pause today.
But I don't see Rust picking up a lot of C devs. Rust is significantly more complex than C. You certainly don't have to use traits and Futures, etc, but you also don't have to use a bunch of features of C++ either and C devs still don't jump to C++.
Zig certainly gets a lot of buzz here, but I haven't personally looked into it. It does certainly sound like it's designed to appeal to the C crowd.
For me cargo, rust-analyzer, and lifetimes/borrow checker just makes programming so much more pleasant.
edit: I dont go overboard on the more advanced features in the rust language, and probably write more C idiomatic code sometimes, but I still prefer using rust over C
But my point of view is quite biased. I am still baffled that people would start new projects in C instead of just using a strict subset of C++, at least: classes, only trivial templates for generics, references, maybe smart pointers depending on what you're doing, but no STL, no exceptions, no inheritance, etc.
Rewriting the world in Rust or other "memory safe" languages is not going to be the silver bullet that is being claimed. In the case of Rust, I am tired of people evangelizing it when it's plagued with issues that undermine its goals and rewrites will bring about more nasty bugs. "Unsafe" code will still need to be written and not being able to use or have stable toolchains to work with is a non-starter for a lot of projects. Until all of these languages have set standards and stable toolchains, hopefully the Rust Foundation will push, these languages are a non-starter.
Do you mind explaining what you mean here? You seem to be alluding to something different than an unstable toolchain which you mention after.
Why spend thousands, if not millions of dollars, on rewriting existing codebases when they could help fixing the existing toolset and make it safer?
I don't get it.
Making C memory-safe by adding extra annotations to allow for static zoning/borrow checking is possible, but the churn involved is probably not much different than rewriting in a different language. See Cyclone for instance if you want to see what such an approach looks like.
On top of that modern languages have facilities that are seriously lacking from C, memory safe or not. C's error handling is atrocious, C doesn't have (standard) destructors, C doesn't have type inference, C doesn't have algebraic types, C doesn't have proper string/collection support etc...
I like C quite a lot, it's probably the language I know best, but it's definitely very outdated. I think in the not so far future it'll be almost entirely replaced outside of legacy applications, hopefully with Rust. I think the watershed moment will be when the Linux kernel is rewritten in an other language, which definitely won't happen overnight but at this point I'm fairly certain that it will happen within a couple of decades.
[DJB disagrees](https://cr.yp.to/djbdns/ipv6mess.html). Quote:
>The IPv6 designers made a fundamental conceptual mistake: they designed the IPv6 address space as an alternative to the IPv4 address space, rather than an extension to the IPv4 address space.
> Either you're C-compatible or you're not.
If we take this black and white approach then we must say that C++ is not C compatible just like Ada isn't C compatible.
The reality is that C++ being broadly a superset of C was integral to its adoption, even though that compatibility isn't perfect. It enabled easy incremental porting of projects.
This is really just a nit though. I do agree with your conclusions.
>0123456789abcdef0123456789abcdef to 192.5.6.30: The client sends a UDP packet to the .com DNS server asking for the address of www.google.com. The client software, intermediate computers, and server software have all been upgraded to handle the client's extended address.
A lot of work in this one "straightforward" step. I know that DJB is a great network engineer, so I'm willing to consider that I'm missing the point here, but at the very least I don't find this particular exposé very convincing.
>If we take this black and white approach then we must say that C++ is not C compatible just like Ada isn't C compatible.
Agreed, in a parent comment I mention C++ being effectively mostly compatible with C. But I can't really imagine how you could do something like that with a memory-safe subset of C. The changes involved are pretty thorough (you basically either need a garbage collector or a borrow checker). Things like raw pointers (especially nullable ones) can't really exist in their current form in a memory safe language.
C++ is not, in any sense, a memory safe language.
To be fair actually using those features of C++ instead of the C legacy is still catching on. Most instructors aren't even teaching it.
Using C in C++ is like using unsafe everywhere in rust - you can do it, but you lose all the advantages.
https://en.wikipedia.org/wiki/No_true_Scotsman
Also can you link to an example of this purported memory safe C++ compiler? Please note for an apples to apples comparison it must reject all memory unsafe non-modern C++ constructs.
std::string_view xx = something_that_returns_a_std_string();
use(xx); //oopsLOL.
In the same idea of "Any sufficiently complicated C program contains an ad hoc, bug-ridden, slow implementation of half of Common Lisp." let's paraphrase
Any sufficiently safe C program contains an ad-hoc (etc.) implementation of the infrastructure needed to make it safe:
- custom string implementation
- custom object/memory management
- custom memory slices and other data structures
Not forgetting the compiler tricks added to try and catch when the program does something stupid
I don't think C can be "fixed" without turning it into a radically different language. At that point, why not simply start from scratch?
This is about language design that makes some languages fundamentally unsafe, not about fixing some bugs or hardening the toolset.
https://compcert.org/ > The CompCert project investigates the formal verification of realistic compilers usable for critical embedded software. Such verified compilers come with a mathematical, machine-checked proof that the generated executable code behaves exactly as prescribed by the semantics of the source program.
https://frama-c.com/ > Frama-C gathers several static and dynamic analysis techniques in a single collaborative framework. The collaborative approach of Frama-C allows static analyzers to build upon the results already computed by other analyzers in the framework.
https://www.cprover.org/cbmc/ > CBMC is a Bounded Model Checker for C and C++ programs. ... CBMC verifies memory safety (which includes array bounds checks and checks for the safe use of pointers), checks for exceptions, checks for various variants of undefined behavior, and user-specified assertions.
So the only chance for advancement are those who propose radical changes, like Rust, or Zig if you want something less complex and "spiritually" closer to C
Clearly, the C committee works hard on improving the language and they are in the process to standardize important features, such as lambda functions, auto, typeof (like C++'s decltype more or less), and so forth.
In my humble opinion, "defer" can mild the buffer / memory overflow, but of course not permanently remedied it 100%; to me though it's better to have such mechanism than have nothing at all to protect you.
https://gustedt.wordpress.com/2020/12/14/a-defer-mechanism-f...
But overall even though I still use C almost daily, I'm a bit skeptical of the committee's direction with these changes. They added half assed generics through macros, now a defer mechanism and a bunch of other stuff that's, IMO, out of scope for C. People have already tried to make a "better C" with all these bells and whistles, it's called C++, and it's still mostly backward compatible with C.
IMO C should be considered to be in "maintenance mode" these days. And again, my IRL job involves writing a lot of it, so it's not like I don't care about it.
I hate the mess that c++ is. It started out with a object based programming paradigm with c like syntax. Then generics/templates were added. Then we have modern c++. It's 3 different languages crammed into one. The c++ committee has a serious case of nifty-itis.
__attribute__((cleanup)) gcc/clang extension already works without extra blocks. GLib has been using it for a while https://blogs.gnome.org/desrt/2015/01/30/g_autoptr/
I see references to "fixing" C and rewriting. An alternative would be to transpile a memory safe, easy to program language such as python3 and generate code in a memory safe language.
Since I don't know which one of these is the best, I have implemented basic (as in fibonacci and a simple class) in several languages.
I hope more effort is spent on writing code in small statically typed languages and generating memory safe code.
That's not just a command line tool but a library, libcurl, with a command line wrapper. That library is easily used from a myriad of memory-safe languages, which is possible in part because it doesn't foist highly opinionated memory representations onto the caller.
But not in a memory unsafe language, please. There's hype and there are lies: https://github.com/rust-lang/rust/issues?q=is%3Aissue+is%3Ao...
But also, stack overflows aren't memory unsafe in modern rust. The issue search you want to find memory safety issues is: https://github.com/rust-lang/rust/issues?q=is%3Aissue+is%3Ao...
Your query searches for type unsafeties. A third query would be for concurrency unsafety: deadlock. All three unsafeties are builtin by design. There's no way to deny it. But they still do repeat the same lies again and again.
Stack overflows being memory safe you can tell your grandma or upper management. Problem was rust was once memory safe, but they ripped out the GC. Now it's just better than C++, but not better than Java or really safe languages. Overhype rarely reflects well on a community.
I agree and I often have to try to make this clear to proponents of "modern" C++/Go.
> Your query searches for type unsafeties.
His query searches for all unsoundness in Rust (including both type- and memory unsoundness). It does not search for unsafeties. Unsoundness can enable unsafety in code written in save Rust. However, unsoundness is fixable w/o redesigning the language (though may be very hard in practice) and usually doesn't cause significant breakage when fixed (in Rust at least). Unsafety in code enabled by fixing unsoundness often disappears when the unoundness is fixed but will result in a compiler error or -lint when the unsoundness is fixed if it doesn't disappear. Unsafety in the language is not fixable w/o major redesign and ecosystem wide breakage. It should be noted that most remaining unsoundness in rustc is suspected to have very low impact in practice and are very hard to fix but they are being steadily fixed.
> A third query would be for concurrency unsafety: deadlock. All three unsafeties are builtin by design. There's no way to deny it. But they still do repeat the same lies again and again.
Deadlocks are theoretically impossible to eliminate in any Turing complete language. In Rust there are often ways to prove your program doesn't contain any deadlocks but indeed some academic research languages (mostly dealing with effect types) makes this easier in more cases. At least in Rust a deadlock results in the lack of stuff happening that you would've preferred happen (which is bad enough), not in UB like in C++ (which is considerably worse).
Memory leaks in Rust are a similar situation to deadlocks although admittedly authors of unsafe Rust need to take some care to ensure it remains similar and not a way that worse memory safety issue could occur. The Rust designers wanted to make it very clear that they consider memory leaks bad but not part of Rust's memory safety guarantees and even went so far as to add a safe function to std that leaks memory (i.e. not as a side-effect, leaking memory is what it is for).
On this note. No language can ever be save in all respects. But save doesn't mean no bugs. It means you have explicit guarantees and enforced contracts. Save rust explicitly guarantees concurrency safety except that deadlocks can occur and the compiler enforces the guarantee unless an unsoundness bug causes the compiler to fail in its explicit contract to enforce it. Similar with memory safety, which is guaranteed (and enforced except sometimes in the precence of unsoundness bugs) but the guarantees explicitly exclude memory leaks. That still leaves a lot less to worry about than in "modern" C++ where absolutely no concurrency- or memory safety is in any way guaranteed nor enforced. Rust is all about eliminating all compromises it can and where the cost/benefit ratio makes sense but to still be pragmatic when and where appropriate.
> Stack overflows being memory safe you can tell your grandma or upper management. Problem was rust was once memory safe, but they ripped out the GC. Now it's just better than C++, but not better than Java or really safe languages. Overhype rarely reflects well on a community.
Stack overflows are memory safe in Rust. Rust's safety guarantees don't say stack overflows will not occur only that they would not cause specific issues if they do occur (like double free or UAF or ...). Again. Sometimes it is possible to prove they cannot occur but in general this is an impossible problem to solve in any Turing complete language.
EDIT: Also: GC's don't prevent stack overflows either. Therefore I don't get your point.
Relying on safe languages isn't going to work as the defect resides in the Memory Management Unit. You would think by now these 'integrated' 'innovators' could come up with a MMU that provides total process isolation.
So, its not enough that anybody that don't want to rewrite millions of lines of code in Rust because its just pure insanity, need to be constantly harassed and forced to feel ashamed (or feel even feel guilt) because "how in the world you did not did this code in Rust"
Now there's a whole INSTITUTION to funnel the scarce resources, not by fixing the security bugs but by forcing others to Rewrite Everything In Rust® or else they will have their popular projects hijacked by highly paid engineers from companies with unlimited financial resources to rewrite the whole thing in Rust leaving the maintainers of the original project eating dust if the don't conceive?
If this is not going to support the source code maintainers to patch the bugs in whatever technology they are using, this is the Rust bullying that used to be just annoying social dynamics, being take into another level.
Daniel Stenberg and libcurl will be exemplary nailed so others feel more inclined to follow or else see their projects being taken out of their hands if they do not consent.
Imagine if you are a open source maintainer with scarce resources, being approached by an institution that promise you to give you some funds, but only if you show a plan to rewrite the whole thing in Rust?
Edit: Just follow the Stenberg tweeter if you are curious https://twitter.com/bagder to see all the pressure this guy have being taken lately.
Now, if the codebase is small enough to allow for scrutiny by 3rd-parties, that point may be moot, but even then, Google may have some ulterior motive which will not immediately become apparent.
So, I'm kind of suspicious of such an initiative.