Oxidizing Ubuntu: adopting Rust utilities by default | Hacker News Reader

Oxidizing Ubuntu: adopting Rust utilities by default

lwn.net

231 pointsjwilk227 comments

> Of course they are considered problematic in Rust. > And leaks are hard to code by accident in Rust. > ...

I enjoyed this gem and its descendants from the comments. What I see instead, commonly, even in big rust projects, is that it's easy to accidentally define something with a longer lifetime than you intend. Some of that happens accidentally (not recognizing the implications of an extra reference here and there when the language frees things based on static detections of being unused). Much more of it happens because the compiler fights against interesting data structures -- e.g., the pattern of allocating a vec as pseudo-RAM, using indices as pseudo-pointers, and never freeing anything till the container itself is unused.

There's nothing wrong with those techniques per se, but the language tends to paint you into a bit of a corner if you're not very good and very careful, so leaks are a fact of life in basically every major Rust project I've seen not written by somebody like BurntSushi, even when that same sort of project would not have a leak in a GC language.

> not recognizing the implications of an extra reference here and there when the language frees things based on static detections of being unused

Maybe I’m misunderstanding, but isn’t the point of the borrow checker to throw errors if a reference “outlives” the memory it references?

How can an extra reference extend the lifetime?

Rust (most of the time, I'm not arguing about failure modes at all right now, let's pretend it's perfect) drops items when you're done with them. If you use the item longer, you have a longer implicit lifetime. The memory it references will also be claimed for longer (the reference does not outlive its corresponding memory).

You only fix that by explicitly considering lifetimes as you write your code -- adding in concrete lifetimes and letting the compiler tell you when you make a mistake (very hard to do as a holistic strategy, so nobody does), or just "git gud" (some people do this, but it takes time, and it's not the norm; you can code at a very high level without developing that particular skill subset, with the nearly inevitable result of "leaky" rust code that's otherwise quite nice).

> drops items when you're done with them.

That is what GC languages do. Too

You can get serious resource leaks in any language if you are careless

What Rust guarantees is you cannot derefernce a null pointer

Sure, but that's not the point of Rust. Many GC'd languages also guarantee that nothing can be null.

I don't know where you got this idea but this is wrong.

> you use the item longer, you have a longer implicit lifetime. The memory it references will also be claimed for longer

This is only true at the stack frame level, you cannot extend a variable beyond that, and there's no difference with a GC in that particular case (a GC will never remove an element for which there's a variable on the stack pointing towards it)

> You only fix that by explicitly considering lifetimes as you write your code

Lifetime parameters don't alter the the behavior, and their are either mandatory if the situation is ambiguous or entirely redundant. And it only works at the function boundaries, where lifetime extension cannot happen anyway. (See above)

Please stop spreading bullshit criticism that have no grounding in reality (Rust has real defects like everything else, but spreading nonesense isn't OK).

Who said anything about extending "variables"?

If you heap allocate something and store a reference to it, Rust will absolutely keep that object alive till all such references are gone. It's not "bullshit [with] no grounding in reality."

That property is, yes, what you would see in a GC language (e.g., Entity Framework back in the day in C# would have a leak if you tried to create short-lived objects from long-lived objects using the built-in DI, since EF had a bug maintaining a strong reference to those objects tied to the lifetime of the outer object). My observation is that people tend to do that much more in Rust than in GC languages because the compiler makes other coding patterns harder.

Rust does not keep variables alive until references are gone. It checks to make sure that any reference lives as long as or shorter than its referent. If the reference is longer lived than the referent, that’s a compile error.

> Who said anything about extending "variables"?

I did, because that's the only thing that can be kept alive longer than needed (till the end of the scope where the variable is defined).

> If you heap allocate something and store a reference to it, Rust will absolutely keep that object alive till all such references are gone.

No it doesn't.

And it shows you have no understanding of how Rust works. A GC will do that, but the borrow checker won't, it cannot, all it can do is to yell at you so that you make your variable live as long as it needs.

See this playground to see what happen when you allocate something on the heap and try return a reference to it: https://play.rust-lang.org/?version=stable&mode=debug&editio...

If you could please stop spreading nonsense about a language you don't even know the basics of, that would be great for everyone.

Can you share a rust playground link with an example of what you are describing?

I still don’t follow how a reference can extend the lifetime of some memory, even on the heap.

Sounds like you are describing a language that uses reference counting to manage memory, and I’m not sure what purpose a borrow checker would serve in a reference counted language.

The claim was "safe", not GC equivalent or memory minimal. You're right, all things have tradeoffs and borrow checker is no different.

The claim was “leaks are hard to code by accident”. I agree with gp that this is false.

Preventing leaks is explicitly not a goal of rust and making lifetimes correct often involves giving up and making unnecessary static lifetimes. I see this all the time in async rpc stream stuff.

I think leaks are indeed harder to code by accident in non-async Rust, which was the original setting Rust was developed on. I wouldn't say it is absolutely hard (that really depends on the architectural complexity), but there seems some truth in that claim.

When a type is 'static, it doesn’t mean it’s leaked for the lifetime of the program. It just means it owns everything it needs and doesn’t borrow anything. It will still free everything it owns when it is freed.

https://doc.rust-lang.org/rust-by-example/scope/lifetime/sta... says:

> As a reference lifetime 'static indicates that the data pointed to by the reference lives for the remaining lifetime of the running program.

What your link says is correct about `&'static T`, a lifetime on a reference. What GP says is correct about `T: 'static`, a type constraint. They both use the "keyword" 'static, but in different senses.

Yes, and IMHO this is one of the biggest design wart of Rust. T: 'unbounded would be much better.

`'unbounded` isn't a bad idea. With my team when someone new to Rust hits the static type constraint I usually tell them an overly simplified "is not a reference" followed up with "The data does not have any constraints (or bounds) on its lifetime that this code needs to worry about."

I don't know if `unbounded` would immediately be understood by someone new to Rust, but it captures that `T: 'static` is a negating assertion. I.e. it largely says what it can't be.

Leaks are as hard to do in Rust as with a GC though.

That is, they aren't impossible and you'll eventually have to fight against one nasty one in your job, but it's far better than without a GC or borrowck.

I don't think that's right - in Rust, cycles of Rc or Arc will leak. In GC languages, cycles are properly deleted.

Cycles with reference-counted pointers are how leaks are caused in Rust and it's specific to Rust, but GCs have their own sources of leaks and in my experience they occur roughly as frequently as cycles in Rust (rarely but not exceptionally either)

The vec pattern has some advantages though, in particular you can often get away with 32 bit indices (instead of 64 bit pointers), making it a little more cache-friendly. I did it for regex ASTs that are supposed to be hash-consed, so never need to die until the whole matcher dies [0].

A more contrived example is Earley items in parser that point inside a grammar rule (all rules are in flat vec) and back into previous parser state - so I have 2 u32 offsets. If I had pointers, I would be tempted to have a pointer to grammar rule, index inside of it, and pointer to previous state [1], so probably 3x the space.

In both cases, pointers would be easier but slower. Annoying that Rust doesn't really let you make the choice though...

[0] https://github.com/microsoft/derivre/blob/main/src/hashcons.... [1] https://github.com/guidance-ai/llguidance/blob/main/parser/s...

By all means. It's a pattern I use all the time, not just in Rust (often getting away with much less than 32-bit indices). You mentioned this and/or alluded to it, but my core complaints are:

1. You don't have much choice in the matter

2. When implementing that strategy, the default (happy path coding) is that you have almost zero choice in when the underlying memory is reclaimed

It's a pattern that doesn't have to be leaky, but especially when you're implementing it in Rust to circumvent borrow-checker limitations, most versions I've seen have been very leaky, not even supporting shrink/resize/reset/... operations to try to at least let the user manually be more careful with leaks.

You should probably use something like https://docs.rs/generational-arena/latest/generational_arena... when you want to do this.

I understand the problem isn’t that the tools exist, it’s that there are Rust users who are not aware of the concept and its evolution over the years, which I’d argue is not a uniquely Rust issue.

Why doesn't anyone rewrite some of these tools in a language that can be compiled to a native binary by GraalVM and benefit of all Java's security guarantees? Would it be too slow? Don't the advantages outweigh the inconvenients?

Rust is trendy, Oracle is the opposite.

Of course this isn't a very nuanced or clever take, but certainly part of the truth

It's also better in just about every way I can think of, except maturity.

You could do similar with the existing C by compiling it to WASM and then compiling that to machine code.

Why use Java when you can use Rust? In all seriousness, Rust is a joy to work with for these kind of tools which typically don't have complex lifetimes or ownership semantics.

On top of that you get better performance and probably smaller binaries. But I would pick Rust over Java for CLI tools just on the strengths of the language itself.

Why go for Java when you can go for .NET? Or Go? .NET seems to perform on par and seems to produce smaller executable, and Go seems to be faster in general.

Personally I don't really care what language common tools are written in (except for programs written in C(++), but I'll gladly use those after they've had a few years to catch most of the inevitable the memory bugs).

I think the difference is that there aren't many languages with projects that actually end up writing full suite replacements. There's a lot of unexpected complexity hidden within simple binaries that need to be implemented in a compatible way so scripts don't explode at you, and that's pretty tedious work. I know of projects in Rust and Zig that intend to be fully compatible, but I don't know if any in Java+GraalVM or Go. I wouldn't pick Zig for a distro until the language hits 1.0, though.

If these projects do exist, someone could probably compare them all in a compatibility and performance matrix to figure out which distribution is the fastest+smallest, but I suspect Rust may just end up winning in both areas there.

Because it’s not a good platform for this at all.

I genuinely don't understand why this is the top comment, as it is almost complete BS and the author confuses the behavior of the borrow checker with the one of a GC (and ironically claim a GC would solve the problem when in fact the problem doesn't exist outside of a GC's world)

There's no confusion between what a borrow checker and a GC do. The borrow checker enforces safety by adding constraints to the set of valid programs so that it can statically know where to alloc/dealloc, among other benefits. A GC dynamically figures out what memory to drop. My claim is that those Rust constraints force you to write your code differently than you otherwise would (since unsafe is frowned upon, this looks like hand-rolled pointers using vecs as one option fairly frequently), which is sometimes good, but for more interesting data structures it encourages people to write leaks which wouldn't exist if you were to write that same program without those constraints and thus wouldn't exist in a GC language.

The borrow checker does not change when things are dropped.

> There's no confusion between what a borrow checker and a GC do

There is, with Rust you cannot extend the lifetime of an object by keeping a reference to it, when this is exactly the kind of things that will happen (and cause leaks) with GC.

> this looks like hand-rolled pointers using vecs as one option fairly frequently

Emphasis mine. Idk where you got the idea that was something frequent, but while it is an option that's on the table in order to manage data with cyclic references and is indeed used by a bunch of crate doing this kind of stuff, it's never something you'd have to use in practice.

(I say that as someone who's been writing Rust for ten years in every possible set-up, from embedded to front-end web, and who's been teaching Rust at university).

> the pattern of allocating a vec as pseudo-RAM, using indices as pseudo-pointers, and never freeing anything till the container itself is unused

Are you talking about hand-rolled arena allocation? I don´t see how a GC language would have a different behaviour as long as you also use arena allocation and you keep a reachable reference.

> There's nothing wrong with those techniques per se, but the language tends to paint you into a bit of a corner if you're not very good and very careful, so leaks are a fact of life in basically every major Rust project I've seen not written by somebody like BurntSushi

If I take 3 random major Rust projects like Serde, Hyper, and Tracing, none of which are written by BurntSushi, your claim is that they all suffer from memory leaks?

And when you put people on a pedestal, they're guaranteed to let you down. :-) https://github.com/BurntSushi/aho-corasick/commit/474393be8d...

I wouldn't be surprised if that style of leak were more prevalent than one would expect. It's pretty subtle. But that link is the only such instance I'm aware of it happening to such a degree in crates I maintain. Maybe there are other instances. This is why I try to use `Box<[T]>` when possible, because you know that can't have extra capacity.

I find the GP's overall argument specious. They lack concrete examples.

Curious why you would call that a memory leak? The memory is still accounted for in the Vec and will get released properly when it deallocates, right? This looks like optimizing memory usage to me, not plugging a leak.

How do you define a leak?

If you sit down and really think about it, I think you'll find that a precise definition of a leak is actually somewhat difficult.

I am nowhere near the first person to make this observation.

I point this out to avoid a big long thread in which we just argue about what the word "leak" means. You could absolutely define "leak" in a way that my example is not a leak. But I prefer a definition in which "leak" does include my example.

I do not care to litigate the definition. If you want to recast my example as a "space" leak instead of a "memory" leak, then I don't object, and I don't think it changes the relevance of my example. (Which I think is absolutely consistent with the context of this thread.) In particular, I don't think "memory leak" in this thread is being used in a very precise manner.

Short of using a different data structure, I'm not sure how you would get out of that one. The claim was that some of these leaks ("leaks"?) could be avoided by using a language with a GC. As far as I know, most modern languages' equivalent of Vec will do exactly the same thing, GC or not.

I was responding to this:

> so leaks are a fact of life in basically every major Rust project I've seen not written by somebody like BurntSushi

And yes, I said I considered their overall argument specious.

I think that regardless of what references you have, Rust frees values at the end of their lexical “scope”.

For example, in the linked code below, x is clearly unused past the first line, but its “Drop” implementation executes after the print statement at the end of the function.

The takeaway is that if you want a value to drop early, just explicitly `drop` it. The borrow checker will make sure you don't have any dangling references.

https://play.rust-lang.org/?version=stable&mode=debug&editio...

In general, I think "lifetimes" only exist in the context of the borrow checker and have no influence on the semantics of Rust code. The language was designed so that the borrow checker pass could be omitted and everything would compile and run identically.

This is correct.

Unless one is writing something where pauses are a no go, even a tiny µs, I don't see a reason for rushing out to affine type systems, or variations thereof.

CLI applications are exactly a case, where it doesn't matter at all, see Inferno and Limbo.

Memory leaks are easy to code by accident in Java, so it must be even worse in Rust.

> This is not symbolic of any pointed move away from GNU components - it's literally just about replacing coreutils with a more modern equivalent. Sure, the license is different, and it's a consideration, but it's by no means a driver in the decision making.

Sorry, don't believe this one bit. I'm very thankful for everything Canonical/Ubuntu did about 20 years ago, but no thanks. This comes from someone who loves Rust and what it made possible. However, freedoms are too important to not treat anything that looks like an attack on them as such.

Which freedoms are being attacked?

Remember when Red Hat tried to lock away the source code to subscribers only? I think they would have preferred not to provide source at all, but the enormous base of GPL code entangled with everything make that impractical.

I think we're really going to miss having the fundamental parts of the system available under strong copyleft, and it will be very hard to go back.

Everything in uutils is under the MIT license. The only thing you're missing is the "viral" nature of the GPL. Nobody in the BSD world seems to be particularly suffering under the thumb of unreleased forks of `chown` and `mkdir`.

Yes, and if BSD was what everyone used for servers instead of Linux, then there would be nothing stopping Oracle from selling you Oracle BSD for a nice subscription price. Of course, they and red hat are already doing this in their own way that gets around the GPL, but a more permissive license would just allow them to do it even more.

Of all the things that constitute a value ad that might allow Oracle or any other company to sell a premium operating system and do away with reciprocal licensing, the core utilities aren’t it. There are already permissively, licensed versions of most of those tools. If people want to champion the GPL and so forth, this is not the hill to die on it’s not really even a hill. It’s a mole hill. That’s not to say that I undervalue the worth of these utilities, but they are not something that people are going to pay premium prices for.

You don’t need to imagine. Apple ships macOS with a BSD user space. The sky has not fallen.

If Linux was the way macOS is, nobody would use it. They'd use macOS, to Apple's great benefit.

I genuinely don't follow your point.

I already prefer macOS over Linux for my desktop, to Apple’s great benefit

There's nothing stopping Oracle from selling us Oracle Linux for a nice subscription price either. The GPL is explicitly not meant to prevent commercialisation of software.

What exactly is this BSD world that you speak of? The one that was mined by corporations for years and whom never gave back anything (except value to Apple shareholders). Compare with Linux which was adopted by corporations and were legally forced to give back. Which made more progress?

> never gave back anything

Apart from Clang/LLVM and WebKit you mean?

Ok in fairness the LGPL license of KHTML was a huge reason the WebKit source was available (for ages it was just reluctant tarball dumps) but I don't think that's the case for LLVM.

I don't think the GPL has really meant corporation's have given back significantly to Linux. There are still plenty of closed source drivers. The main motivation to give back is the extremely high cost of maintaining out-of-tree modifications (e.g. Android's binder).

LLVM has spawned probably a dozen vendor supplied toolchains that you never see the source for, best of luck figuring out why your program didn't compile correctly, or adding support for something other than the tools/languages provided.

The compiler and toolchain really are going the wrong direction with LLVM in terms of open source.

Yes Apple's clang weird ass versioning causes all sorts of problems too

Some proprietary-friendly compiler would've appeared in one way or another at some point. I'd rather such at least have some amount of openness. Better than many crappy from-scratch completely-proprietary vendor compilers at least.

And hey some of the work does get upstreamed, which is probably a good choice for the vendor anyway if the target in question isn't intended to be immediately thrown in the thrash and forgotten.

>Ok in fairness the LGPL license of KHTML was a huge reason the WebKit source was available (for ages it was just reluctant tarball dumps) but I don't think that's the case for LLVM.

It was originally supposed to. https://lists.gnu.org/archive/html/emacs-devel/2015-02/msg00...

The last 2 paragraphs of this interview with David Chisnall really made me think differently about that: https://lobste.rs/s/ttr8op/lobsters_interview_with_david_chi...

In particular:

> I think the GPL has led to fairly noticeable increase in the amount of proprietary software in the world as companies that would happily adopt a BSDL component decide to create an in-house proprietary version rather than adopt a GPL’d component.

It also aligns with my experience: my company couldn’t find an LZO compression library that wasn’t GPL’d, so the decision was between implementing one in-house or cutting the feature. We ended up restricting use of the feature to in-house use only, but opening up our core source code was never an option.

If there had been a permissive license option available, we would’ve likely donated (as we do to several other dependencies), and would’ve contributed any fixes back (because that’s easier to explain to customers than “here’s our patched version”).

I've seen this personally. Google does not allow GPL code to imported into the massive monorepo. If I needed a library and the only good OSS option was GPL, I'd write it from scratch instead. I'd also usually not open source those things if this wasn't part of an already open source project, because it wasn't worth the effort.

So, the company wants us to trust them - trust that they will not take advantage off the BSDL to do things the GPL would prohibit them from doing. And we're supposed to trust them because doing so means that there would be more "hands on deck" (e.g. working on the now-canonical LZO compression library that everyone uses because it is BSDL).

Sorry, trust has been broken too often in these scenarios, and the benefits of lots more people working on the same library are not entirely clear.

I understand that many companies don't want to be a part of the pool of software licensed under the GPL - that is their right. But don't try to spin this "if only BSDL was the common one, there'd be a much bigger utilization of a different pool of software". That might even be true, but it would come with the caveat that tivoization would always be an option, which for some of us is something more significant.

It's not about trust, it's about the fact that 10% of something, or even the chance of 10% of something, is better than guaranteed 100% of nothing. You're not being taken advantage of if someone uses a BSD license the way it's supposed to be used.

If I put two choices in front of someone, the binary option of copyleft vs proprietary and they'll always go proprietary, or I give them something permissive and there's at least the chance they contribute back, the second option is strictly better. It's sort of the equivalent of having a wealth tax that raises no revenue, I'd rather have you here contribute something than move abroad and get nothing, even if the benefit is more indirect

It's not 100% of nothing. If the choice is proprietary because they legally can't freeload off FOSS due to it being GPL, the company in question still needs to pay a developer to write the alternative. That's still a positive, and nothing lost. Permissive licenses just grant to freedom to freeload, nothing more.

I don’t think it would’ve in our case, at least. I actually quite like our OSS policy: basically, we don’t want to be in the business of maintaining forks, so all changes should try to get upstreamed if at all possible. It‘s good client service as well: when our clients ask about the libraries we use, we’d much rather be able to tell them “the latest public version” than “here’s our fork”.

We also want to engage with and donate to the projects we use, mostly out of risk management: if they go unmaintained that’s bad for us.

If you truly want to upstream changes to libraries then I don’t see why lgpl code isn’t a choice for you. Use it in your proprietary code if you want, no changes no foul. If you make changes you just need to upstream or make the source available.

> If you truly want to upstream changes to libraries then I don’t see why lgpl code isn’t a choice for you.

Read LGPL a little more carefully. LGPL requires you to use it in a particular manner that lets any user swap out your version for their version, which basically means you're only allowed to dynamically link it as a shared library. Even if you make no changes to the library!

> that lets any user swap out your version for their version, which basically means you're only allowed to dynamically link it as a shared library

I had thought that the dynamic linking requirement was the only option according to the licence, but apparently not. According to 6.a. of v2 and 4.d.0. of v3 of the LGPL, it would be enough to give the user access to the source/object code of the main application and the compilation scripts under non-open-source terms, so that they can recompile and statically link it against their own version of the library.

I think the wealth tax analogy is fantastic because some people genuinely hate rich people so much that they’d rather they left the country.

Yes there exists now more proprietary software but you wanted to create proprietary software with it anyway so this kind of is what parent was talking about that some people don't want.

Partly related. Would you have paid if the project had offered a paid non GPL licence?

We (engineers) actually wanted to for another GPL’d project! But because they didn’t have a CLA, the lawyers wouldn’t sign off on it — they decided that the main/current maintainer didn’t have the rights to relicense it for us.

We probably would’ve for LZO too; not sure why that fell through.

> because they didn’t have a CLA, the lawyers wouldn’t sign off on it — they decided that the main/current maintainer didn’t have the rights to relicense it for us.

How would the legal argument be any different for MIT/etc. licensed software in that case? Would the lawyers sign off on using MIT-licensed software without a CLA? Wouldn't they make the argument that the provenance of the software and therefore its licensing is not solid? Seems like the only thing that matters is who has the right to offer a license to the software, not what the license is.

They just wanted the CLA to support the (paid) relicensing.

I think the reasoning (as it was explained to me) was that when people made their original contributions, they were agreeing to the license at that time (in this case GPL, but for other projects MIT). But the other contributors never agreed that the main maintainer could relicense their contributions for a fee.

The upshot was that we went with an in-house fully-proprietary alternative. More expensive, probably lower quality.

I quite agree here, but to go a step further here:

Lots of people express this fear that BSD-licensing of components will lead to a world where all the big proprietary software just locks everything down. But if you actually work with large software, you end up finding out that it's actually a lot of work to maintain your own fork, and so you start pushing to get more things upstreamed. Because, after all, if your code is private, it's your task to fix it if somebody breaks it, but if it's upstreamed, it's their task to do it.

An interesting datapoint is LLVM/Clang, where, yes, lots of companies have their own proprietary forks of Clang. But for the most part, those companies still upstream gobs of stuff: most of the biggest contributors to the project are the people with the proprietary forks. Furthermore, as I understand it, basically everybody relying on EDG has, or is in the process of, worked to move off of it into Clang. Which, if EDG dies, means that the permissively-license Clang has done more to kill off proprietary compilers than copyleft-licensed GCC has.

The best defense against proprietary software is to make it too expensive for proprietary software to compete, and using a permissive license means you can trick the companies making proprietary software into helping you do that.

> An interesting datapoint is LLVM/Clang

It's also an example of the GPL not preventing corporations from building (and releasing under a more permissive license) non-GPL software.

Before Apple built Clang, they made a GCC plugin[0] which dumped the AST from the GCC frontend, and fed it into the LLVM backend. Sure, they published the source of their GCC modifications under the GPL, but that was nothing compared to the entirely novel backend, which was released under a far more permissive license.

Meanwhile, Stallman handcuffed GNU projects like Emacs for years[1] by refusing to expose GCC's AST in an effort to prevent something like this from happening.

[0]https://dragonegg.llvm.org

[1]https://lists.gnu.org/archive/html/emacs-devel/2015-01/msg00...

> Before Apple built Clang, they made a GCC plugin[0] which dumped the AST from the GCC frontend, and fed it into the LLVM backend.

But they didn't! Dragonegg is a GCC plugin, which didn't exist when work started on Clang--GCC plugins date to GCC 4.5, released in 2010, which is the same year that Clang became self-hosting.

The timeline is rather like this: LLVM originally relied on a hacked-up version of gcc 4.2 called llvm-gcc to create LLVM IR from source code. When GCC added support for plugins, there was the attempt to move that frontend to a more reliable framework, and that was Dragonegg. As far as I'm aware, Apple never contributed to the Dragonegg project itself. Dragonegg didn't last that long; by 2014, the project was completely dead. And for most of its life, the biggest interest in Dragonegg was to have some modicum of support for Fortran.

> Meanwhile, Stallman handcuffed GNU projects like Emacs for years[1] by refusing to expose GCC's AST in an effort to prevent something like this from happening.

By the time of that message, RMS's worst nightmare had come true with Dragonegg... only for Dragonegg to have already died due to a lack of demand for it.

You're right, I got Dragonegg mixed up with llvm-gcc. But my point stands – the GPL was completely ineffective at preventing someone from using it to bootstrap a new compiler infrastructure under a more permissive license.

What he said:

> the GPL has led to fairly noticeable increase in the amount of proprietary software in the world

What he did:

> opening up our core source code was never an option.

It's companies that decide that open sourcing their software is 'never an option'–even the LZO compression part of it–that lead to a 'noticeable increase in the amount of proprietary software in the world'. They are just using the GPL bogeyman as a thin excuse for their own decisions.

His insinuation that if you just MIT your code so that big tech companies can use it, and if they end up needing help with it they might even hire you, seems super sleazy and borderline exploitive especially in the current job market.

Red Hat is probably the biggest single provider of GPL software in the world. They consistently choose the GPL for software they author and release. They founded and fund the Fedora Project themselves explicitly to be a community driven distro independent of their commercial efforts. They provide the full source to their products even though a huge percentage of the code is MIT, BSD, or Apache and does not require them to. They provide everything required to produce the "aggregate" full product even though the GPL does not require them to.

I will go on record now with the prediction that, if uutils catches on, Red Hat will be one of the last companies to move away from GNU. They are probably the largest contributor to glibc and GCC.

What evidence do you have that they would have "preferred not to provide source at all"? Because their is a mountain of evidence otherwise.

As an individual user, Red Hat will give you a free license to their flagship product. Then they will tell you how to download every line of code for it. I don't have a license. I do not use Red Hat (other than occasionally using the also totally free RHEL9 container).

"Remember when Red Hat tried to lock away the source code to subscribers only?"

I do not remember them changing the policy you are talking about. If you think they did, it highlights how overblown the response was at the time. The biggest impact of the Red Hat change is that Alma Linux is now a better project that can actually innovate and contribute.

You defended Red Hat, but it's Blue Hat now. They may not have damaged their legacy yet, but we're on the lookout.

IBM is one of the biggest open source contributors in the world. What are you looking out for exactly?

Did you miss this one?

https://arstechnica.com/information-technology/2023/06/red-h...

Again, be careful not to attribute past RH actions with IBM. Yes, IBM was very helpful in the 90s and later, but you can't fully trust a corporation past the next financial statement, and certainly not past a leadership change or three.

Did I miss Red Hat applying the license of their software with the rights given to them by said license? No, I did not, I supported it. The GPL is perfectly compatible with commerce and a profitable business model and this example shows exactly why. What would you prefer–companies that open source software bleed money and get taken advantage of by others? I don't. I want to see a healthy open source ecosystem that preserves user freedoms while balancing that with profitability. The GPL perfectly embodies this ideal.

Fragmede above describes the details on why we keep a healthy distance from IBM.

Provide the link please! Don't make me hunt through the thread to figure out what you're talking about lol

https://news.ycombinator.com/item?id=43443062

As he says, it's just business. And IBM isn't near the top on my list of big tech companies to avoid. However they are subject to the same enshittification pressures as the rest. Their history is not all rosy either.

Two things stick out for me.

Red hat provided the source to their kernel with the letter but not the spirit of the law, to impeed Oracle Linux. We'd (Ksplice) get the changes from Redhat version to version as one giant patch file, with all the changes mixed together. There's no chance in hell their internal kernel developers worked that way, but the source was there, so the letter of the GPL was satisfied.

The other thing they did was close the source repos. First you had to be a subscriber to get the binaries, and thus by the GPL, you could get the source. However, if you shared that source code with anyone else and they caught you, they wouldn't let you renew your license the next year, so you couldn't get the newer version, binaries OR source. This also isn't technically against the rules of the GPL, but it's hard to see that as being in line with the spirit of it.

It's hard to see Redhat as the benevolent GPL open source hug supplier with real world business practices like that. Not that I fault them for it, it's just business.

Oracle Linux — that was the bugfree no exploit version they touted?

Unbreakable Kernel.

That isn't something Canonical could do just because the software has a permissive license instead of a copyleft license. They can do that if they own the copyright (which I can't imagine they do), but if they own the copyright they can do anything they want and nobody has a say in it. So again it has nothing to do with license.

The absolute most that Canonical can do is start making a fork of the software where they don't distribute the source code. But in that case, the original code is still right there. No freedom is lost at any point.

The original code does not matter. What matters is the code for the binaries you are running. That is the entire point, and that specific garantee has far reaching consequences, even if you apparently can't imagine them.

Red Hat has been more consistently pro GPL and pro open source and invests more money into it than any other company.

Them putting the downloads of their enterprise OS behind a login screen (student and solo developer licenses can STILL be got for free) is something I 100% understand and am kind of sympathetic about.

I know that's out of like with how some people define free software, but I've always had ethical issues with the way people would redistribute Red Hats work for free. Just because it's legal doesn't make it ethical.

User freedoms. Those don't exist under the so called "more permissive" licenses.

The GPL ensures the software will always be free. "Permissive" licences don't. Permissive licences permit anyone to create a derivative version that isn't free. No source code, and it will be under copyright.

If there's one thing people need to understand it's that corporations will always strive to take as much as they can and give back as little as they can. It's incredibly naive to think they won't do it again after that's exactly what every corporation has done for the past several centuries. Give an inch and they'll take a mile. You can't seriously think companies like Microsoft and Oracle aren't already salivating at the thought of the community rewriting GNU/Linux in a permissive licence?

It's not that this particular case is "freedoms being attacked". It's that freedoms are constantly and relentlessly under attack and we can't let our guard down for one moment. GPL is our protection and voluntarily stepping out of it would be suicide.

You mean the Microsoft which not that long ago releases the proprietary .Net code under mit? It's not all going one way, even in large corps. Windows also ships with wsl now - what would they even gain now from non-gnu Linux?

They did both those things because it’s the only way they can compete. You really think they would open source any of that stuff if developers weren’t leaving their platform in droves? WSL is them just exploiting foss to make up for their own shortcomings.

.NET case was mostly a rewrite, many components were rewritten from scratch (sooner or later). Roslyn was a compiler-rewrite to make C# self-hosted with, what I understand, intention to make it OSS from the beginning. .NET (then Core) started with RyuJIT. AFAIK RyuJIT also mainly exists for .NET only, where-as .NET Framework uses older, different compiler codebase.

But yes, there were additional licensing assurances provided so as to not create problems for Novell/Mono - it was always treated positively.

Ubuntu lost its way a long time ago. Pushing Snap by default, ads in the Unity Dash search, ads in the terminal... Unity itself was a misstep, and around the time the distro started going downhill for me.

I don't follow it much these days, but nothing really surprises me from Canonical anymore. They leech off free software just like any other corporation.

I just realized this today. When trying to upgrade an EOL-ed Ubuntu machine using 'do-release-upgrade', it completely shat the bed. Now I'm in search of an alternative distro that has good GPU support, rolling releases, no systemd. Maybe OpenSUSE?

If there are any SV billionaires out there, can you fund CUDA on OpenBSD please? :-P

No systemd really limits the options (including that OpenSUSE uses it, so I'm not sure how to read your comment). Maybe Artix Linux? That's rolling release systemd-free, though I don't know about graphics drivers. I would have suggested Pop!_OS, with has an awful name but is basically like Ubuntu without the bad stuff and excellent graphics drivers, but it has systemd and isn't rolling so YMMV.

Oops, my read was that you can opt out of systemd with OpenSUSE without much trouble.

I don't know about the good GPU support and is not a rolling release, but Devuan is a systemd-free Debian derivative you might be interested with

You can run Trisquel Mate, but you need to set the propietary kernel and CUDA on your own. You can use the Xanmod kernel, the headers and the prop NV installer, but you will be less free on freedoom. With Intel and OpenCL (or whatever intel uses today), you might loss performance, but you gain compatibility and you are not bound to Intel X86, in case you want to try further Power9 based stations.

I use Nonguix with a RTX2080 and am very happy with that. Similarly to Nix it's marvellous, but not for the faint of heart.

Whatever happened to the whole debate around Ubuntu including ZFS modules by default in Ubuntu? At the time it originally got proposed, I feel like I remember basically everyone other than Canonical agreeing that this wasn't allowed by the licenses of Linux and ZFS, but they did it anyways, and from what I can tell, they basically got away with it?

Unless I'm remembering it wrong, I'm honestly not surprised that they might just be less worried about licensing in general after that; maybe this is the software licensing equivalent of "too big to fail"?

> from what I can tell, they basically got away with it?

Or they correctly interpreted the law?

Few of us care to admit how this "legal consensus" is mostly BS. Lots of evidence to the contrary, but, like politics these days, it sure is an appealing fantasy?

I don't pretend to have any legal expertise on the matter. All I know is that I remember reading stuff that made me think there was a consensus that this wasn't allowed by the licenses, and that they did it and I didn't notice any consequences. Maybe it was BS, but I think it's also fair to argue that there's quite a lot of licensing terms in software that haven't been fully established via precedent in the courts yet and instead mostly ends up being followed out of risk aversion rather than anyone being totally 100% sure how a court would rule (which isn't something people can necessarily predict on _any_ given issue, rather than something specific to software licenses).

If the common consensus was BS, I actually don't think that detracts from the larger point I was trying to make much; I probably distracted from it too much with my over-enthusiastic wordplay at the end, but my original intent was to convey that Canonical might just intentionally have decided that they aren't concerned with the types of licensing issues that people are discussing here with regards to coreutils. If my theory is correct, it's due to sound legal judgment or intentional non-compliance isn't something I have any strong insight into; I mostly just thought it was worth suggesting that what people here think about it might not really matter all that much because it seems unlikely to me that they'll either be convinced to change their plans or suffer significant negative consequences from going forward with what they've decided, regardless of whether it's right or wrong.

>Whatever happened to the whole debate around Ubuntu including ZFS modules by default in Ubuntu? At the time it originally got proposed, I feel like I remember basically everyone other than Canonical agreeing that this wasn't allowed by the licenses of Linux and ZFS

I think your recollection here is pretty colored, or else you were only reading stuff from one particular bubble. My own recollection is that there were very compelling arguments to the contrary, ones that I agreed with. Like, foundational to the concept of civil court cases is "standing" which is almost invariably down to a party suffering damages. Courts aren't a venue (at least in the US in general) for solving debates of abstract philosophy, that's what civil society is about, they're for applying the state's general monopoly on violence in order to rectify harms. Party A claims they were damaged by Party B (statutory, actual or both) in a way that violates the law. The remedy if they win is generally that they are made whole via money as best as possible or, rarely, via actual performance by Party B.

But if open source license GPL code is distributed with open source license CDDL code, with full access the source of both available, who is losing money on that? Who is getting damaged? What rights are being lost? Who has standing to sue over it? The idea that it's banned at all isn't tested legally anyway afaik, but more fundamentally if there is no harm then there simply isn't any court case. Say you saghm decided tomorrow that you were really steamed about ZFS being included with Ubuntu and decided to get going on your lawsuit on Monday, when the court asks "how much money did that cost you and what do you want us to do about it" what would your answer be, even putting aside "what's your theory of law on this one".

>and from what I can tell, they basically got away with it?

Well, that would be expected if there just isn't any case there right?

Your interpretation doesn't seem compatible with (my understanding of) how IP licensing as a part of contract law works. If I distribute copyrighted music via torrent, the RIAA isn't going to waste time finding out if the recipients were planning to buy CDs otherwise. If you receive software under a license that is too restrictive for you, then there is always the option to pay the rightholder to grant you a more permissive license in exchange for money.

(I don't approve of this system, I am merely describing what I see as the status quo.)

> (I don't approve of this system, I am merely describing what I see as the status quo.)

As has been said, being a FOSS "advocate" may put you on many sides of any copyright issue.

> RIAA isn't going to waste time finding out if the recipients were planning to buy CDs otherwise. If you receive software under a license that is too restrictive for you, then there is always the option to pay the rightholder to grant you a more permissive license in exchange for money.

Your comparison fails to understand how different these claims are. First, those CDs have a price on the tin. That is -- this claim would have "actual damages" which can be readily determined. Second, you're speculating a new license for either Linux or OpenZFS is possible, but I don't think anyone imagines either is readily available given the number of rights holders. If Oracle was offering to relicense (BTW only its portion of the ZFS source) for a known and reasonable amount $X, like your CDs, then your comparison might make more sense (but read on, the offer of another license does not preclude a finding of non-infringement).

Thankfully, of course, copyright law provides for statutory damages where actual damages cannot readily be proven, as here. But those nominal/statutory damages divided up among Linux rights holders would be almost insignificant, and therefore this poster above is likely correct -- this may not be a legally cognizable injury.

While I may agree re: this poster's damages analysis, I'm not even sure that's the best form of the ZFS argument. IMHO where anyone creates a functional software interface, whether it be via game console's cart or a kernel module, fair use allows one to create to software and use that interface. Period. Your license can say whatever it wants. Your license can say "all your software now belongs to use" and IMHO it is completely unenforceable. IMHO the courts have favored software interoperability and fair use for years.

For example:

    In Sega Enterprises, Ltd. v. Accolade, Inc., the court denied copyright protection for “functional requirements for compatibility with the Genesis console . . . .” Under this approach, the right of the kernel module author to create a compatible module overrides any nominal copyright infringement created when that author creates static or dynamic links to kernel code. Sega’s Genesis console had no public API whatsoever, stable or otherwise, yet the court still denied protection to these functional elements. Regardless of the status of the API or system interface required for compatibility, any parts of a program that a developer must copy — such as kernel headers, definition files, variables, or mandatory Linux kernel function calls — in order to create a Linux-compatible kernel module would not receive copyright protection.[0]

The case is much stronger re: ZFS because the court explicitly protected reverse engineering in Sega, whereas here the Linux kernel devs have published the code to their interface.

Note, Sega even offered Accolade a license (just as we discussed the potential for an alternative license above) that would have allowed Accolade to create Sega-compatible games. Notwithstanding this offer, the court declined to find copyright infringement in Accolade’s creation of Sega-compatible games without a license.

[0]: https://www.networkworld.com/article/836041/smb-encouraging-...

> As has been said, being a FOSS "advocate" may put you on many sides of any copyright issue.

Who's an advocate here?

> Your comparison fails to understand how different these claims are.

Comparisons don't understand anything. I mean to draw attention to aspects shared by two different things.

> I don't think anyone imagines either is readily available given the number of rights holders

Me neither, but I don't think that's pertinent. Why do you?

Finally, you make an argument about interfaces, but I think the issue here is the use of CDDL licensed code, rather than the (re-)implementation of an interface. To me it seems like the finding in Sega v Accolade aren't relevant here.

> I mean to draw attention to aspects shared by two different things.

Yes, I understand you meant to reason by analogy, and I told you that I think your analogy is flawed. When you reason by analogy, perhaps you should be prepared for someone to say, "That analogy doesn't make sense in this context." Mostly because -- your analogy doesn't make sense in this context. One, is an actual damages case, and the other is statutory damages case. When we are strictly discussing damages, your comparison is an apples to oranges comparison.

You said:

>>> If you receive software under a license that is too restrictive for you, then there is always the option to pay the rightholder (sic) to grant you a more permissive license

You suggest "there is always the option to pay the rightholder (sic) to grant you a more permissive license" and I'm saying that it's impossible in this circumstance? There is no one stop shop for the licensee. There is no 50 stop shop.

In your scenario, where we are to imagine an incompatibility exists, you also imagine a theoretical alternative license would be relevant to a court. And I'm saying, just as relevant, is the actual impossibility of obtaining the consent of either all ZFS licensors or all Linux licensors.

We've created effectively perpetually licensed software via our shared use of FOSS. And now one party we imagine saying -- actually these two works will be incompatible forever. You think a court won't think that's wild?

More significantly, this entire line of argument has an angels on the head of a pin quality. The GPL and the CDDL are the licenses that have already been chosen by Canonical, because Canonical didn't, like you, read an incompatibility into this CDDL-GPL combination. So -- fair enough that Canonical didn't feel the need to cast about for some theoretical alternative license, if it doesn't feel that the CDDL and the GPL are incompatible.

> I think the issue here is the use of CDDL licensed code

As discussed -- I don't think it matters much how the code is licensed. If it makes it easier, imagine the CDDL sourced code is actually a closed source, proprietary filesystem. So -- Linux provides a module interface, but it could be any interface. Just as in Sega, IMHO even proprietary code can use GPL licensed code, which provides an interface, for the purposes of interoperability. Why? Because "fair use".

I don't agree with any of what you've said. Thanks for the argument.

Ubuntu stopped being Ubuntu when it stopped being brown.

i am not that concerned about more permissively licensed versions of what are effectively commodities (like coreutils). I am still very interested in keeping something like the kernel as GPL since it doesn't have a real substitute.

Why don't you believe it?

From my perspective, it seems imminently plausible. Who cares about the licensing?

But I'm interested in what leads you to the opposite conclusion.

> Who cares about the licencing.

Well, for one, the author of the Rust based utils cared enough to change it (or rather to not re-adopt GPL, but IMO that's the same ting). Why shouldn't we care about the licencing?

Yeah I was being tongue in cheek. (Or just overly dismissive I guess.)

I know people do care about licensing, but it's also plausible that it isn't what any given project is most focused on.

I see no problem with Rust. The problem here is the licensing. The new project is proprietary-compatible.

So what? There's zero interest in proprietary extensions to coreutils. This isn't Linux or GCC; they're just basic command line utilities.

Also anyone that really cared could use the BSD versions anyway.

The uutils project has the right goals - 1:1 compatibility with GNU coreutils to the extent that any difference in functionality is a bug.

The first comment on LWN is about a bug in the more(1) from uutils. I checked out that code, found some oddities (like doing stat() on a path to check if it exists, right before open()ing it) and went to check against how GNU coreutils does it.

Turns out coreutils does not do it at all, because more(1) is from util-linux. ta-dam.

Not sure why you are downvoted but I’m curious why. I appreciated this tidbit, thanks.

And? How does that do it?

I checked. It fopen's the file and then fstat's it. So it isn't vulnerable to TOCTOU.

However the TOCTOU is completely benign here. It's just an extra check before Rust opens the file so if you were to try to "exploit" it the only thing that would happen is you get a different error message.

> if you were to try to "exploit" it the only thing that would happen is you get a different error message

Can't reproduce this. If I do

    sudo strace -e inject=stat:delay_exit=30s:when=2 ./coreutils more foo

on one terminal and

    rm foo
    ln -s /etc/passwd foo

on another, I can see the contents of /etc/passwd on the first one.

Uhm yeah well now try

  ln -s /etc/passwd foo
  sudo more foo

OMG! The original version is "vulnerable" too!

(I did not know about that strace feature though - that is very cool, thanks!)

I thought I was gonna see an error because the stat'd file and the opened one didn't match. How disappointing

No, it just means you're checking the wrong file so you might not get the nice error message from the initial check.

It's still not going to do anything incorrect when it comes to actually open the file though.

TOCTOU only matters when the check is actually checking something important. Here it isn't; it's just giving nicer error messages. If you bypass it you just lose those nice error messages.

So it isn't a problem here.

> > Klode said that it was a bad idea to allow users to select between Rust and non-Rust implementations on a per-command level, as it would make the resulting systems hard to support.

Wouldn't this imply that these aren't actually 1:1 replacements?

It would imply not all of the replacements are complete and/or bug free yet, and the article gives examples of just that earlier in the text, but it would not imply the replacements lack the goal of being 1:1 replacements like GP said.

This would seem to be one of the driving factors in the creation of oxidizr: to allow testing how ready the components are to be drop in replacements with easy recourse. You can read more about that in this section of the linked blog post by jnsgruk https://discourse.ubuntu.com/t/carefully-but-purposefully-ox...

  uutils aims to be a drop-in replacement for the GNU utils. Differences with GNU are treated as bugs.

-- https://github.com/uutils/coreutils?tab=readme-ov-file#goals

the more you know

I think this is an excellent point that a lot of people are missing.

There are a lot of GNU utils. For now, Ubuntu is only taking on the ones that are actually in coreutils. Those are the ones that uutils considers "production ready".

That's funny because just today I was reading the Rust docs for std::fs::File and saw a warning about that exact TOCTOU error, and had to fix my code to not hit it

std::fs is generally not a good API with the complete lack of directory handles.

>The uutils project has the right goals

Their goals are to _replace GPL's code_. It's a subtle attack on Free Software by providing corporations a workaround so they don't have to abide by the terms of the license.

Nobody seriously doubts the efficiency or safety of the GNU Coreutils. They've been battle tested for over 30 years and whatever safety or optimization risks are left are negligible.

I hate to admit it, because I don't particularly like Rust, but I'm slowly coming around to the idea it should replace things.

The major change is a recent comment I made where I was musing about a future where LLMs actually write most of the code. That isn't a foregone conclusion but the recent improvements in coding LLMs suggests it isn't as far off a future as I once considered.

My thought was simple: if I am using a LLM to do the significant portion of generating code, how does that change what I think about the programming language I am using? Does the criteria I use to select the language change?

Upon reflection, Rust is probably the best candidate. It has strictness in ways that other languages do not have. And if the LLM is paying most of the cost of keeping the types and lifetimes in order, what do I care if the syntax is ugly? As long as I can read the output of the LLM and verify it (code review it) then I actually want the strictness. I want the most statically analyzable code possible since I have low trust in the LLM. The fact that Rust is also, ahem, blazingly fast, is icing on the cake.

As an aside to this aside, I was also thinking about modular kernels, like Minix. I wonder if there is a world where we take a userland like the one Ubuntu is trying and apply it to Minix. And then slowly replace the os modules with ones written in Rust. I think the modularity of something like Minix might be an advantage, but that might just be because I am completely naïve.

I've had similar thoughts as well. Rust or C would also be the ideal candidate from an energy consumption point of view, as they consume way less energy than Python, which is the language that LLMs most often default to in my experience.

However it's unlikely LLMs will generate a lot of Rust code. All these LLMs are generating is the most likely next token, based on what they've been trained on. And in their training set, it's likely there is massively more Python and JS code out there than Rust, simply because those are way more popular languages. So with Rust, it's more likely to hallucinate and make mistakes than with Python where the path is much better trodden

In practice, it is not what happens. I've been doing AI assisted Rust for some time, and it is very convincing that this is the way. I expect 6mo to 1y to be basically fully automated.

Rust has tons of code out there, and quality code. Different from js or Python that has an abundance of low quality to pure garbage code.

same here. I think rust + llm combo is unbeatable.

However it is much easier to statically analyse rust and rust has compile time validations compared to an interpreted language like python. This makes it easier to produce code with write compile fix loop from an agentic llm in rust than python.

In my experience llms are not particularly bad with rust compared to python, though I've only toyed with them minimally.

While I'm lucky enough not to ever have had to spend a while trying to refactor/fix LLM-produced code, I've definitely had a much easier time trying to do major refactors to fix larger issues with human-written Rust code than any other language I've tried it with. The compiler obviously doesn't catch everything, but the types of things it _can_ catch seem to be very common to encounter when trying to make cascading changes to a codebase to fix the types of issues I've seen refactors attempt to address, and my instinct is that these might also be the types of things that end up needing to be fixed in LLM-produced code.

Elsewhere in this thread someone mentioned LLMs producing poor performing code that does stuff like collect iterators far too often and pointed out that fixing this sort of thing often requires significant Rust expertise, and I don't disagree with that. However, my impression is that it still ends up being less work for an experienced Rust programmer to claw back some memory from poor code than for someone similarly experienced in something other than Rust working in a code base in their own language. I've seen issues when three or four Go engineers spend weeks trying to track down and reduce memory overhead of slices due to the fact that it's not always easy to tell whether a given slice owns its own memory on the heap or is referencing a different one; looking for everywhere that `collect` is called (or another type known to heap allocate) is comparatively a lot simpler. Maybe Go being a lot simpler than Rust could make LLMs produce code that's so much better that it makes it easier for humans to debug and fix, but at least for some types of issues, my experience has been the opposite with Go code written by humans, so I wouldn't feel particularly confident about making that prediction at this point in time.

It’s interesting it took you this long to substitute “junior coder” with LLM. The implicit safety is just as applicable to teams of human error prone devs.

The rust produced by LLMs is quite bad. It’s overly verbose (misses combinators) and often subtly wrong (swallows errors on result types when it shouldn’t). A single errant collect or clone call can destroy your performance and LLMs sprinkle them for no reason.

Unless you are experienced in rust, you have zero ability to catch the kind of mistakes LLMs make producing rust code.

Is this deficiency likely to persist long-term as LLMs grow more powerful?

I think one issue is less extant Rust code to train on. As more Rust is written and published, LLMs should in theory get better.

I heard most code today is written by LLMs. I hope that's not true, but if it is, the future improvements will be slow (barring something like, actually teaching them to read the docs).

I just had an LLM (something 3.7) propose a 100 line implementation and after I stared at things for a while, I reduced it to one. I’m sure I’m in the minority of not just accepting the 100 line addition.

Could you elaborate? What were those lines doing?

Lots of the lines were safely dereferencing things in JS where you could just do ?., putting try/catch blocks around that (each time!) and calling into things in a way that a single line would have sufficed. I almost took a screenshot for the lols.

Be honest, it was one line that should have been broken into 3 for readability. This vicious calumny must stop, only 97% of LLM output is rubbish and not the 99% you claim.

> Unless you are experienced in rust, you have zero ability to catch the kind of mistakes LLMs make producing rust code.

I'd say it's on par with other languages then... LLM are roughly 95% correct on code generation but that's not nearly enough for using them.

And spending time finding which 5% is looking good but actually wrong is a frustrating experience.

Those programs are making different kind of mistakes than humans and I find them much harder to spot.

LLM truly provides the garbage in, garbage out experience

This is the worst kind of busy-work. Rewriting something for the sake of it is terrible practice.

There's a lot of refreshing energy amongst Rust coders eager to produce better, more modern command-line tools. That's amazing, and a real benefit to everyone. But rewriting simple, decades-old, perfectly functional tools, with the explicit goal of not improving them is just a self-absorbed hobby. And a dangerous one at that - any new code will contain bugs, and the best way to avoid that is not to write code if you don't have to.

It's not for the sake of it - it's to have MIT-licensed (able to be propriotized) coreutils. GPL is a thorn in the side to anyone who wishes to EEE it.

That isn't the motivation. It's about "resilience, performance and maintainability".

I doubt they'll get noticeably better performance (the main GNU tools are already very optimised). I'm not sure they really lack in resilience. And I don't think memory safety is a big factor here.

Maintenance is definitely a big plus though. It will be much nicer to work on modern Rust code than ancient C.

To clarify, you think the obvious benefit that is something several organizations obviously want isn't the motivation, but improvements to 3 things that probably won't be improved are the real motivations?

How much maintenance do the tools need currently, and how does it compare to the effort?

i am surprised by the implication

If you look a bit deeper, this project actually cares deeply about its license, and is going out of its way to choose the license it is using, ignore complaints, and avoid ending up GPL.

https://www.youtube.com/watch?v=5qTyyMyU2hQ

In an interview with FOSS Weekly, Sylvestre Ledru (the main developer, who curiously has a background working on Debian and Firefox, before ending up getting seduced by the clang/LLVM ecosystem), firmly states "it is not about security".

This is far from OK. Entire point is to have GPL not to run away from it. Again canonical shows they didn't get the point. Anyway Im staying on gnu coreutils as i see no benefit and zero reason for switch.

Yeah because non GPL coreutils don't exist.

toybox and the BSDs already exist for those wanting permissively licensed utilities, the GPL is not a major motivator.

They do, but they're minimal versions, not one-for-one compatible with the GNU versions.

Every proprietary Linux OS (such as most Android phones) uses busybox, at a slight disadvantage to them, because they can't handle using GPL3 coreutils. Now they'll use the MIT-licensed drop-in replacement coreutils.

Previously:

https://news.ycombinator.com/item?id=43360969

https://news.ycombinator.com/item?id=43353240

Amazing, another reason for avoiding Ubuntu.

Many of these utilities have had logic errors in them, you can find them on GitHub issues. sudo, for example, allowed you to bypass through some way I cannot remember.

And I bet you they are not a replacement for GNU utilities, i.e. have less features, and are possibly not optimized either, and perhaps they even have different (or less) flags/options.

I have written a lot of Bash scripts, I wonder how well (if at all) would work on Ubuntu with the Rust utilities.

What an ignorant comment. The uutils README on GitHub explicitly states in the first section that every discrepancy wrt GNU utilities is considered a bug.

It is a comment coming from pragmatism, not ignorance.

At any rate, feel free to look around here: https://github.com/uutils/coreutils/issues?q=is%3Aissue%20st...

It is NOT a replacement of GNU coreutils AT ALL, as of yet.

Granted under "Goals" it says "uutils aims to be a drop-in replacement for the GNU utils. Differences with GNU are treated as bugs.", but please look around the opened (and closed) issues (since they should tell you about the major, yet simple logic bugs that has occurred). It is definitely not ready, and I am not sure when and if it will be ready.

FWIW the README also says: "some options might be missing or different behavior might be experienced.".

Future will tell, but right now, it is an extremely bad idea to replace GNU coreutils with Rust's uutils. Do you think otherwise? Elaborate as to why if so.

Your first sentence is totally unnecessary and relies on an uncharitable reading of what they said. It's entirely possible that they knew that a goal of the project was to treat every discrepancy as a bug and were indicating that they were of the opinion that there were plenty of bugs left and no evidence that they would ever be all squashed.

Also, while we're slinging the README around, here's the first paragraph:

> uutils coreutils is a cross-platform reimplementation of the GNU coreutils in Rust. While all programs have been implemented, some options might be missing or different behavior might be experienced.

The "ignorant comment" above was actually just pointing out the thing that the developers thought was second-most-important for people to know—right after the fact that it's a reimplementation.

> uutils aims to be a drop-in replacement for the GNU utils. Differences with GNU are treated as bugs.

> some options might be missing or different behavior might be experienced.

Both of these statements cannot be true at the same time.

They trivially can: "aims to be" means "in the future they will be this way," not "they are presently this way." That's what "differences are bugs" means, that you may currently experience incorrect behavior, but that it is intended to be fixed in the future.

According to GitHub issues and the project's current status, that future is going to be a long time from now, so why would anyone NOW replace GNU coreutils with this project? It makes no sense. I hope they will at least give users the option to choose, but then again, it does not affect me, I do not use Ubuntu, nor will I ever touch it.

Canonical making light of licensing again, no surprise here.

This has ulterior motive written all over it. Best outcome: nothing happens.

I have a queston: Why do they need their utilities to be written in Rust? Most utilities do not have to face the network, and most don't need to have root privileges. Heck, most of them are one shot programs and don't even need to free memory by the programmer. They could exit and return all the memory space back to the OS.

> Most utilities do not have to face the network

True, but code from those utilities may eventually be used in the network (for example, through copied functionality and shared libraries). Also, a creative pipeline may actually involve them (think of the Unix philosophy.)

Eventually plain C has to die or be relegated to unavoidable places like in the assembly cases, even if Rust is not the best alternative after all.

This is based on a common fallacy that people believe about CVEs vs. what actually gets exploited. I would go back and read the papers that got you so aggressively on board the memory safety train one more time and see if you can't detect motivated reasoning.

What is the fallacy, specifically?

If you have an argument to make, make it. This isn't a classroom and you're not an instructor, assigning homework to people who disagree with you isn't an effective argumentation technique.

Because things need to keep evolving, we see great new ideas coming from tools which add progress bar, make colorful output, have knowledge about .git structure etc. Current state of things is quite stalled, in my opinion.

It's funny git structure support is often cited as a good feature to have, baked in a tool that should outlive git itself.

But the explicit goal of this project is not to evolve, but to rewrite 1:1.

Also making progress bars and colors does not require Rust, it's equally possible in C, C++, Zig, Go, Swift, etc.

It already adds a couple new features, like a progress bar to a few utilities.

Directly facing the network is just one way that a program may come process untrusted inputs.

Because people take input from the Internet and run it through them.

My long hope is that some day, someone starts to replace the perl-based Debian infrastructure with Rust (or really, anything).

I did a decent bit of mod_perl, love & respect perl, but here in 2025, it's pretty terrifying to me that Debian's main dependency is a pretty sizable perl runtime. That there seems to be very little that will budge this. The whole ecosystem is built atop a language that very few people have interest in, that has much less interest & activity than other modern alternatives.

It's fine if we start switching the user env over to more popular Rust based utilities. But what I really want is for the OS itself to modernize, to start getting away from it's ancient seemingly-unshakeable legacy. Nothing against Perl, but I'd love to see this family of OSes move beyond being Perl only.

There are non-Perl-based replacements for basically all the system utilities written in Perl in Debian in other distributions, but as things are currently, the Debian versions written in Perl are still unmatched.

All you are doing is setting people up for more permanent bash-dash disasters. Isn't there something more useful you could be doing with your time? Say making drag&drop work in snaps (another unforced disaster)?

The "security" in Wayland, Snaps and Flatpaks are starting to irk me. Macros aren't working, clipboard forgets my copy, drag n' drop is flimsy - AppImages like Cursor can't even open on vanilla Ubuntu anymore. These "features" no one asked for should be opt-in.

In a way they are opt-in, as in you opt in by using Ubuntu and opt out by using something else.

Yep, exactly. The whole thing is just theater. I hate it. I know I can just use X11 (and I do), but the already small appetite for developing for linux will be stretched even more thin in the coming years and we will probably only have Wayland apps. The bright side however, is that Wayland could make itself so utterly garbage that people just ignore making apps for it (I'm not counting apps targeting just kde/gnome, which anyway diverge far enough from wayland protocols to warrant being mentioned separately). So we would have X11/KDE/Gnome support for linux apps. That's probably the best case scenario.

Do you think app permissions in Android and iOS apps are not helpful? There isn't really a reason that the permission models make sense on mobile but not desktop. Desktop applications are inherently more trustworthy. The fact the security features gets in your way is more of a product finesse problem which generally plagues open source projects in general. The technology is not conceptually flawed.

> Desktop applications are inherently more trustworthy

Why? If 30 years of Windows being the most internet connected OS has proven anything, it's that 99% of desktop executable should be deleted before they make it through the disk write cache.

Mobile platforms show that mandatory sandboxes are a massive bonus to preventing malware. Desktop operating systems refuse to go that route because of the risk of breaking existing tools, which is why nobody bothers to actually use sandboxing APIs, which is why many third party sandboxing attempts keep breaking.

There's no reason why desktops can't exchange files through virtual file systems or dedicated sharing APIs like on mobile. My calculator doesn't need camera access and my music player can do without biometrics. The desktop is stuck in the 1990's permission model (or 1970s, for Unix-likes) because that's what people have gotten used to.

My experience with the Steam Deck is that Flatpak sandboxing on a read-only system image works absolutely fine. Things would work better if desktop applications would bother standardising and following existing standards, but the technology works. Sure, it's not good enough if you're a Linux kernel developer, but the 0.01% of users who are affected by those limitations shouldn't limit everyone else.

Some desktop users may think themselves to be too smart to get hacked, especially on Linux, but their desktops are not that different from smartphones.

I had a typo, it should have been aren't. I agree with you.

I agree, they should rewrite the kernel in Rust

yea thats called a completely different project

> they should rewrite the kernel in Rust

I adore Rust

Please don't!

Can we get on with innovating on the backs of what went before, not reinvent it...

Has someone built Busybox in Rust yet? That would be good for embedded.

technically uutils/coreutils should suffice for this goal still as it can build into a single-binary tool a la busybox (iirc that's the default actually)

Busybox has very basic implementations of the tools though, to keep the size in check.

This is on brand for Canonical.

Bleeding edge beta (alpha?) software put in the core of their system

Reminds me of when they switched Gnome out for Ubuntu One, no way back if you did 'aptitude full-upgrade'.

It was a buggy leaky pos

Again, dear friends...

It looks like a corporate project with all the required hype and lingo. Ubuntu was nice around version 10, it went downhill with spyware, snap, etc.

"There are countless examples in the open source community of tools being re-engineered, and re-imagined using tools and practices that have only relatively recently become available."

Recently? You can use Ada or OCaml, which is a better Rust. You can use C with formal proofs and not constantly rewrite everything.

But that of course is the point. No rewrites, no easy money for the corporate OSS "developers".

I love the ml language I'm learning and consider it better _than_ any other language I use, but I'm puzzled by "ada or ocaml, which is a better rust"

Never written rust, only looked at snippets on blogs, but it visually looks so different to ml

Am I being fooled by superficial syntactical differences?

Could you explain?

Using Rust is a political solution to deskill a generation of coders to replace higher-cost labor with lower

The push for Rust isn't just a technical decision - it's a calculated economic strategy. By forcing rewrites of stable systems in a new language, companies effectively reset the clock on developer experience and expertise.

When a C/C++ codebase with 30 years of institutional knowledge gets rewritten in Rust, senior developers with decades of experience suddenly compete on more equal footing with juniors who just graduated. Your 15 years of C++ optimization knowledge? Now worth less than a 22-year-old's six months of Rust bootcamp training.

This isn't about memory safety - it's about labor costs. Companies call it "modernization" while quietly erasing the premium they'd otherwise pay for experience. The technical arguments serve as perfect cover for what's really happening: deliberately manufacturing a scenario where they can replace $250K senior engineers with $80K juniors.

The pattern is familiar to anyone who's watched other industries. Create artificial obsolescence of existing skills, then exploit the resulting chaos to reset salary expectations and eliminate the leverage that comes with specialized knowledge.

This is why language transitions always seem to coincide with hiring freezes and "restructuring." It's never been about technical superiority - it's about breaking labor's bargaining power.

That's an interesting conspiracy theory I haven't heard before. However, I doubt the investment into rewriting large projects makes any financial sense even if the goal is to kick out the experts. Rewrites take years, even if the junior does it for cheaper, and migrations are an existential business risk.

If those 15 years of C++ optimization don't provide an advantage over what a 22 year old with half a year of bootcamp can provide for a quarter of the price, then maybe it really is time to stop doing things in C++.

The unfortunate truth of programming as a profession is that you need to move with the times. If C++ is on its way out (which I highly doubt), it's time to learn Rust. Most Visual Basic programmers moved on, as did most Fortran programmers, and many C++ programmers when Java hit the scene.

When you're making this about labour's bargaining power, you're assuming that young labour is a threat to experience labour. Perhaps that's the case to some extent, as the shortage of people with programming skills that has led to the ridiculously high wages for the software industry is slowly evaporating, but gatekeeping the industry isn't some kind of pro-labour move, it's just selfish.

It's more like replacing 4 C++ software people with 1-2 Rust people, because there's half as much work when you aren't constantly coding foot guns.

The rest really is tin foil hat stuff. I'll believe it if I see it, until then its hard to see Rust as a deskilling. The language requires a different skill is all.

GNU is free to adopt better tools than C/C++. I adore gnu but I refuse to support blatant and obstinant idiocy. Using the license to extort people into supporting C/C++ is just scummy behavior. What the fuck were you doing the last ten years that this wasn't a gnu project? Instead we got fucking systemd and a dozen more knockoffs of the shittiest os on earth

You have the right to not use GNU utilities or GNU/Linux either, you know.

No one's forcing you.

Rust and "memory safety" is the current form of corporate authoritarianism. It's ironic that its most vocal advocates are also likely to be the ones complaining about the authoritarianism of the current administration.

Memory safety isn't so corporate, AFAIK. I think it's just a current programmer fad, like pearl's dwim, python's "pythonic", JavaScript having 5 different libraries for every conceivable one-line function, Haskell's um ... I never got Haskell, etc.

Every time a language starts taking off, people want to rewrite everything in their language. Rust just has the performance and a few advantages that make it arguably better than C for the things C does, which is a bit cool I guess.

It is corporate, because they realised that things like jailbreaks and rooting could be stopped easily by increasing "security" this way.

Reply on news.ycombinator.com