Curl removes experimental HTTP back end in Rust | Hacker News Reader

Curl removes experimental HTTP back end in Rust

daniel.haxx.se

168 pointshnarn69 comments

> We have shipped hyper support in curl labeled EXPERIMENTAL for several years by now, hoping to attract attention and trigger the experimental spirit in users out there. Seeing so many people seem to want more memory-safety, surely the users would come?

This analysis seems off. Memory safety isn't a feature users will explicitly ask for or opt-in to. What users want is software that isn't buggy, and they just sort of ambiently expect it. A "Rust backend" is pretty meaningless to users, a bit like expecting users to pass a --dont-be-buggy flag to every invocation of the CLI tool.

As a user, I'd have questions like: if this backend is better then why isn't it the default? Why do I have to make a decision at all? I just want an HTTP library/tool that doesn't let me get hacked, how it achieves that is none of my concern really. Whether that's done using Rust or not isn't something that should appear in the user interface.

If you want to read a bit of Rust community criticism into the Article, I think it would not be too far off to read this as “There are lots of people claiming writing things in Rust provides better software due to memory safety, but in practice users did not care and were perfectly content with the C implementation” (presumably because the latter has no noticeable bugs).

I also interpreted the article a bit that “users” here refers to developers that make use of libcurl, that would overall like to ensure that the application that they are writing does not segfault or similar. It seems plausible to assume that some of the Rust goodwill would translate to developers that may also try to migrate their overall code to Rust to try out that backend, and potentially contribute improvements. The Linux kernel and Git for example seem to attract a lot of attention of people that want to work on migrations towards more Rust.

Hyper can't be the default if it doesn't pass the test suite (that's the bare minimum). And after years there weren't enough contributors to fix all the remaining bugs.

I would agree that the curl project itself should have put more effort in fixing those bugs and eventually remove the C HTTP/1 backend, not the Rust code. That's what happened in Firefox for example, and other projects that adopted Rust incrementally.

I think this has to be put against the larger context of Rust… fans that push very hard for the entire world to be rewritten in Rust. Just look at how many Rust projects where their front page / README has the first line as "written in Rust" as if that's the number 1 feature a user should care about. As such I think there was an outsized voice of re-writting cURL in Rust and whatnot as occasionally there will be buffer overflow bugs that admittedly would not have existed if the code in question was written in Rust. That's why there was the experiment to add a new backend using Hyper because in theory they should not be prone to the same class of memory bugs.

As for why not making it the default, the post already made it clear? The new backend did not pass all the tests and therefore could not possibly be the default backend until that was fixed and turned out fixing the last few issues were quite hard. cURL has also supported different backends for a while now. For HTTP/3 for example you can swap out the backend for something else. This is like driving a car. If you literally don't care at all, just use the stock tires/engines and it will drive fine. But sometimes you may care and swap out the stock components for something else that suits your needs, and cURL allows that. cURL / libcurl gives a stable ABI as a glue but it's intentional that you can swap out the backend.

ah, the classic "--bugs" / "--no-bugs" flag

--bugs / --other-bugs

I don't really like the idea of mixing languages in projects, be it curl or the linux kernel, doesn't matter. If an already established and proven language was already being used for such a long time successfully, that should remain the language of the project. Forks or entirely new / independent projects may be created with whatever new language is being hyped up, and that is totally fine / should be the way to go. In the end, more harm will come from forcing these new languages into long-established projects than benefits.

> If an already established and proven language was already being used for such a long time successfully, that should remain the language of the project.

I guess it depends on what exactly you mean by "successfully". For example, while C is used for Linux and Linux is quite successful by most measures, that doesn't necessarily meant that there aren't areas where C isn't successful - in this case, a significant reason there's so much interest in Rust is precisely because there has not been success in producing reliably memory-safe code in C.

I think "forcing" might be a bit of a strong word for what's going on as well, but that's neither here nor there.

I agree, Rust indeed has an important role as a new language for writing memory-safe code, but to repeat what I meant, I like to see entirely new projects built with Rust from the ground up, not bolting it onto existing C or C++ projects just because Rust. In the long run it will cause fragmentation and increase complexity to such a point that it could destabilize the original project in its entirety. Build new things with Rust if you like, keep existing C projects as C projects, that is all.

> not bolting it onto existing C or C++ projects just because Rust.

I feel "just because Rust" leans a bit towards the overly-dismissive side. I'd imagine Linus isn't interested in potentially adding support for Rust "just because Rust" - presumably he thinks there are potential concrete upsides that Rust offers over C that are worth expending time and effort to investigate despite the potential complications from introducing a new language.

> In the long run it will cause fragmentation and increase complexity to such a point that it could destabilize the original project in its entirety.

This sounds scary and all, but I'm admittedly a bit skeptical about this argument. Why would introducing Rust support necessarily lead to the outcome you describe? Are there examples of such an outcome happening to other projects?

I agree that API stability should be more important than what it currently is for many libraries. If you change the language, that's a pretty major API breakage, unless you provide some way to still use it the same way. But if something was a C library and suddenly it's only a Rust crate, then I'm not sure I'd even call it the same name/project anymore.

Personally I wish libraries would just never remove/change anything from their API, that we'd be able to have some sort of immutable API. Only additions could be made, or you can fork it into another project if you want a different API.

No? Adding a Rust backend does not change the API. It just calls into the Rust backend via C bindings. If all you care is ABI stability from the user side none of this would break.

cURL already supports other Rust-written backends as the article pointed out. See https://everything.curl.dev/internals/backends.html.

That is the dream of the Open–closed principle in SOLID and Postal's law like in RFC 1122.

While these ideas are good defaults, unfortunately they lead to unmaintainable, monolithic, insecure code. One needs to make decisions on tradeoffs based on current needs and balance on how to move forward.

Remember how Tor anonymity was broken due to Postal's law? Or how many encryption downgrade attacks have resulted in breaches?

IMHO the better option, but a hard sell, is to focus on externalization and methods like ports and adapters and DDD to help you write interfaces that are more likely to be stable, while still having polices in place to help you move forward when needed.

The chaos report from the 90's and many many more studies since have showed that trying to get software projects perfect from the beginning is impossible.

There are just to many unknowns and trying to plan for them all is impossible.

In some situations like REST it is easier so that breaking changes can be released in a new API version, with a deprecation schedule.

With other things like graphql, ffi etc... it becomes far harder.

If changing the language results in API breakage, you weren't following the inversion principle.

Even with language features, what you call a 'stack' is an abstract data type, not a concrete implementation. While there may be very strong disintegration drivers like performance or language limitations that drive you leaking implementation details or limit your ability to confirm to the ADT while maintaining the other properties you want, that is not specifically the concrete implementation that results in that breaking.

ADT's like lists, stacks, or queues are defined by their semantics from the user's perspective.

Obviously Rust's design decisions may make duplicating the semantics while gaining the advantage of the switch challenging or possibly impossible.

While not really the target audience for either language, I would prefer zig's crashing behavior to hyper using unsafe evaluations of macros as an example.

https://github.com/hyperium/hyper/blob/30f2961e89eb306780d85...

Both are better than c's problems like `x[4]` actually being `*x` though. (IMHO)

The point being is, do target a forever API as a default ideal, but don't stick to that ideal too strongly or it will result in an outcome that is far worse for anything more complicated than say UNIX file operations which are simple enough to do so.

We are in a industry where the 'best' option rarely if ever exists and we almost always have to choose the least worst options.

If some external party can figure out how to refactor without breaking the ADT/API semantics, you shouldn't care except be impressed that they avoided leaking their implementation details to you.

> While not really the target audience for either language, I would prefer zig's crashing behavior to hyper using unsafe evaluations of macros as an example.

> https://github.com/hyperium/hyper/blob/30f2961e89eb306780d85...

This looks basically like a port of something like Kotlin or C#'s ?. operator - the macro checks whether the pointer is null, returns err if it is, and dereferences the pointer otherwise. The part highlighted in the link looks like it's part of the macro implementation - there don't seem to be any "direct" uses of that particular rule in calling code and it's only invoked via the other rules in that macro definition. I think that should make that bit null-safe at least.

Lifetime safety might be a bit trickier, especially since that bit is part of hyper's C API so idk if there's anything Rust (or Zig, for that matter) could do to ensure the pointer is still valid when dereferenced.

> If an already established and proven language was already being used for such a long time successfully, that should remain the language of the project.

Any reason why? On paper C doesn't seem to offer any benefit that Rust doesn't, nor Rust any harm that C doesn't.

Polyglot projects are harder to reason about, harder to onboard to, have more complex tooling, and can have weird interaction bugs.

Normally I'd agree, but rust integration with C is so tight that this is of reduced concern.

What counts as a project in your case? Would you oppose the idea of modules within a project mixing languages? I believe curl is over 500,000 lines of code so in the case of forks a progressive rewrite, module by module seems a lot more achievable than porting everything all at once

I view even the Linux kernel as a project in this sense, as I equally don't like the idea of mixing in Rust modules into the Linux kernel which was so far (and in my opinion should still remain) a C-only codebase. I'm ok with writing an OS in Rust, but make a separate new project based on Rust, don't mix Rust into an already established project like the Linux kernel.

I think there was little traction in curl, because Rust users can just use hyper directly.

https://lib.rs/curl is used in 1000 packages, https://lib.rs/hyper is in 21,000 packages.

Curl is big and supports lots of protocols, but Rust doesn't need a one-stop-shop for all of them. HTTPS covers majority of uses, and Rust has separate packages for mail, ftp, websockets, etc.

Hmm, wasn't it the other way round? Curl using a Rust library (hyper) instead of Rust programs using curl?

Disclaimer: Just reading TFA, not an active Rust programmer.

Yes, but curl-with-hyper needed Rust programmers to finish and maintain the integration with the C codebase, and couldn't find anyone interested enough, which I assume is because Rust users don't need curl.

It sounds like a lack of funding (i.e. the grant ran out) was the real issue. Given how high profile curl is, this raises the question of how sustainable rewrite-in-rust efforts driven by grants (or other short-term funding) are, if they don't have an existing rust community to take advantage of the grant.

This wasn't a rewrite-in-Rust effort, and I think that's the problem. Nothing valuable from curl has been rewritten in Rust.

Only some existing Rust code has been added to curl, but the Rust ecosystem already has a better, safer way of using that code.

Curl is not planning to ever require Rust, so the rewrites are limited only to optional components, and can't fully guarantee that the code is safe. The Rust components are required to expose an unsafe unchecked C interface for the rest of curl. C compilers are unable to enforce the safety invariants of the interface, like the Rust compiler would in a program fully written in Rust.

Probably I misread the original announcement, but I got the impression this was a pilot to adding more rust to curl (and is exactly what a rewrite would start to look like)?

The original announcement explicitly said it's not a rewrite, multiple times: https://daniel.haxx.se/blog/2020/10/09/rust-in-curl-with-hyp...

> A rewrite of curl to another language is not considered

> This is not converting curl to Rust.

> Don’t be fooled into believing that we are getting rid of C in curl by taking this step.

Curl plans to live and die with C: https://daniel.haxx.se/blog/2017/03/27/curl-is-c/

This has been reaffirmed recently in https://daniel.haxx.se/blog/2024/08/06/libcurl-is-24-years-o...

> There was never any consideration to use another language than C for the library […] Not then, not now.

Not only curl is rejecting possibility of being rewritten in Rust, it's also committed to supporting completely-Rust-free curl, because they pride themselves on supporting a lot of retro/niche platforms that Rust doesn't exist on.

I think you're right. Curl is a rich source of C era vulnerabilities (memory safety, weak typing, etc.). Anyone using Rust has already decided they don't want anything to do with that.

Except when they use `unsafe`, which they do, a lot.

Can you back up that claim? In my experience, the vast majority of Rust is safe, or built on communally-audited safe abstractions over unsafe code.

Using communally audited abstractions over unsafe code means you’re using unsafe a lot.

If there was some way to prove that the abstraction is safe, then that would be fine. But the inadequacy of communal auditing is the reason why C has security issues.

The area of Rust code that is unsafe is much, much smaller than the amount in equivalent C code, making it much more tractable to audit. I won't pretend that it's perfect, but it's not remotely comparable to C.

There’s no easy bound on the set of code you’d have to audit to confirm that even one use of unsafe is in fact safe.

It's literally THE unsafe part of the code. It's the only part of code that can invoke UB.

  fn do_something() {
      unsafe { ... }
  }

  // Somewhere in the program
  do_something();

Doesn't matter where "do_something" is used and how much. The only possibly problematic part of this code is the unsafe block. You only audit it.

But if you can manually identify an invariant inside an abstraction it can greatly improve performance for callers/users, additionally, tools like Kani use comprehensible macros to facilitate automatically proving safety of `unsafe` code. Not to mention built in linting, package management, docs, and FP that rust/std provides. Lots has been said about unsafe rust, but the most basic libc tools require the whole cascade of upstream callers to check safety, it's basically backwards from the ground up from a resources and an outcomes perspective.

Use of unsafe is very rare (except for FFI to C where it's unavoidable). I've written tens of thousands of lines of Rust and used `unsafe` exactly once.

Exactly the same experience : my toy compiler project has 10477 lines of Rust code, and there is only one line with unsafe:

  let is_tty = unsafe { libc::isatty(libc::STDERR_FILENO) } != 0;

Here's the source, just in case: https://github.com/burjui/rambo/

In fact, there are exactly two "unsafe" blocks in all of my Rust projects, and the second one is not even needed anymore because of the language and ecosystem improvements, but the project is basically abandoned, so I'm probably not gonna fix it. There's just no need for unsafe in the vast majority of code.

I don't know where Rust critics get their statistics; probably from picking their noses, judging by their arguments. Most don't seem to even have read the official docs, the bare minimum required to form any conclusions at all about the language. I guess they don't read much Rust code and think there is no way we can write even semi-decently performing high-level code without resorting to unsafe hacks or calling C, because that's the way it's done in other languages.

yeah, rust is mainly for developers,and curl are for sysadmins and their derivatives.

_lib_curl

https://curl.se/libcurl/

https://curl.se/docs/companies.html

I have worked in other large companies that use libcurl and they aren't even listed above. It's pretty much the de facto way to do HTTP requests in C-land unless you really want to write your own backend for some reason. The world still primarily runs on C/C++, not Rust.

I could see rust catching on as a shell scripting language too, honestly. Well documented, typed abstractions over shell utilities is really quite nice.

I disagree. I would hate to wrangle a borrow checker just to write a simple script.

Why it's always the borrow checker that bugs the haters? You don't even see it work most of the time, because instead of using references everywhere, like most C++ers are conditioned to do, not only you can move values (doesn't necessarily mean actually performing memory operations), but also the compiler checks that you don't use the variable from which the value has been moved. And even if you do use references everywhere, they don't usually cause any problems, unless you decide to put more than one reference in a often used struct. I often use both shared (RO) and exclusive (RW) references as function arguments and rarely have to specify lifetimes manually, because automatic lifetime elision done by the compiler is sufficient in most cases.

Who said I’m a hater? That was a very aggressive response. All I said is that I think Rust is not a good language for a scripting system. In scripting, I’m not always writing something “correct”, but good enough. Mutability everywhere helps do so, but the borrow checker gets in the way of that.

I've reached some mechanical sympathy with it, I think.

You get some intuition for which methods to use, many structs have a pretty breadthy pallet, and then you can mostly just ignore it; quickly write your algorithm and let the compiler walk you through any oversights with the way you may have coded processing tasks.

Also helps to know that the docs support search by type signature since functions operate on or return owned, borrowed, or mutably borrowed, and generally just think of borrowed as a pointer for heuristic reasons.

It makes more sense than I do, usually.

That is quite the arbitrary distinction. There are plenty of sysadmins that code and/or hold compsci degrees.

Not if you ask meta.stackoverflow.com

One thing I've wished for in curl is for the backends ("handlers") to be more pluggable. This would solve a packaging problem we have in Fedora where we'd prefer to distribute a core curl with backends packaged as subpackages, so that programs that only need (eg) http support would only need to depend on curl-handler-http and curl-core. At the moment the actual solution is to compile separate curl and curl-minimal packages where curl-minimal removes the lesser-used backends, and you have to choose, system-wide, which one to install.

This plus a stable ABI for handlers would also solve the writing backends in other languages problem because another team could contribute a Rust / hyper handler separately.

(I'm sure Daniel has solid reasons for not doing this, not to mention that it's a bunch of work which no one is volunteering to do.)

The curl-minimal solution sounds more transparent and understandable to users, on the other hand.

The problem with curl-minimal is if you install any program that needs full-fat curl, then you're out of luck if you want to keep the minimal install.

With separate curl handlers you'd also be able to find out what protocols your system is configured to access with a command like this [simulated]:

  $ rpm -qa | grep curl-handler- | sort
  curl-handler-ftp
  curl-handler-http
  curl-handler-https
  curl-handler-smb

and similar RPM commands might be used to find out what applications need a particular protocol.

Yep, but still as a user I'd prefer the simpler way, since it will be more clear what's happening ("ok, curl-minimal is being replaced by the full curl when I install this package"). And there will be fewer little splinter packages.

I wonder if there's a place for an additional level of abstraction here, so that the user would just see something like "subcomponent X of the curl package is being installed"...

As a rust dev I wish we had more choice of http libraries then hyper. I have a lot of respect for the author but I consider this library a hot mess of over complicated abstraction. If you tried to use hyper recently you know what I mean, just to do anything you need 4 sister crates. The internals of hyper are both hard to read and opiniated in way that makes me swear heavily every time I have to touch it.

hyper more like a build block rather than a tool you can directly use, maybe you just should use other libraries which built on the hyper

Yeah, I'm using the curl wrapper crate for a Rust re-write of a C++ HTTP monitoring app I'm re-doing, because there's nothing I can find in the Rust ecosystem that even comes close to what libCurl exposes in terms of functionality and control (i.e. things like all the different timing numbers).

The wrapper crate hasn't got everything exposed for the Easy interface (I'll probably be submitting PRs to expose some missing stuff at some point), but seems good enough for me.

Only issue really I've found so far is that heap fragmentation seems orders of magnitude higher than the C++ version, and so memory useage grows quite alarmingly, but from heaptrack, that all seems to be from within openssl, with things like this (a rediculous number of heap allocations per request):

https://github.com/openssl/openssl/issues/14837

but I'm not sure why the older C++ version doesn't suffer from the same issues as it's doing the same thing with HTTPS requests and is also using openssl, so I still need to look into that...

Currently doing:

unsafe { libc::malloc_trim(0); }

every 5 minutes keeps the memory use under control, but again, I don't need to do that in the C++ version, so there's some difference somewhere in the libcurl stack between the two...

There is hundreds if not thousands of rust crates for http to choose between. `reqwest` is just one of many other popular alternatives, all built for different sorts of tradeoffs.

Most of them are built on top of hyper, including reqwest [0] (not a bad thing, they're just still hyper).

I personally wouldn't even consider hyper as an HTTP client for use in most projects, but reqwest adds a much more approachable API on top of hyper.

[0] https://github.com/seanmonstar/reqwest/blob/master/Cargo.tom...

And to get a simple https request you end up with about 200 crates in your tree

This is an (unfortunate?) part of the Rust ecosystem. Reminds me of the JS/NPM mess. I think it is a bit better in the Rust world, but I’m still undecided if it is acceptable to have so many crate dependencies for conceptually simple programs.

No, there are smaller crates out there that enable a simple https request with far less crates.

Much of that is due to not needing Tokio & co.

Yes.

Part of the problem is there are so many, which do you choose?

I found one (I forget the name to protect the guilty) but my simple use case broke it.

I reached out to the authors, very helpful, I think they fixed it.

But by then I had moved on (time is money)

I sucked it up and went with importing all the tokio stuff I did not need (I absolutely did not need, almost no one does)

Rust has painted itself into a corner. A really very good idea at its core, almost all of it is really terrific, but some corner cases like these make it very hard to take advantage of it

Deciding which one to try isn't hard at all: `ureq` is the de-facto small HTTP library that you'd reach for, and you're likely to find people referencing it within 2 minutes of Googling. The only thing `ureq` was missing that's common (ish) was a happy eyeballs implementation, and they've recently handled it.

This just isn't a significant problem at all.

> Deciding which one to try isn't hard at all

Nonsense.

If you are steeped in HTTPS clients for Rust, maybe, but I can tell you from personal experience that coming fresh to https://crates.io/search?q=https%20client and friends is overwhelming

A few minutes in Google would've saved you that confusion, same way it works for any other package manager.

> We had to rethink and reassess several implementation details when we aligned HTTP behavior with hyper. libcurl parses and handles HTTP stricter now. Better.

That would be interesting to dig in deeper. Wish they did that in the article. Or linked an article where they do.

They removed a backend. Rustls and QUIC are still an option

I'm curious for more details about which tests were hard to align and why

imo rust is a great language, but the leadership/project setup is very untrustworthy.

The systemic security problems of cargo package distribution opens the door for state-sponsored threat actors to performing supply chain attacks. And when asked about these issues the only people who will comment using their real identities are based in China.

The security issues in the rust development experience are well-documented since several years, so my summary at [1] was no news to them.

But if you offer things like rustfmt, rustdoc, and a nice-to-use package distribution platform like cargo.io you should be a bit more concered with security imo.

Their lax attitude to these issues and substantial corporate interests not only from US-based but also China-based companies might be good for funding but for me raises many questions.

[1] https://github.com/bf/rust-security-problems

I agree! I actually wish crates.io or even the Rust language as a whole would embrace the "sans-io" mindset. Crates which are "sans-io should simply be "no_std + alloc". That would suffice for 99% of crates. Ideally, to be fully "sans-io", you'd only be able to depend on other "sans-io" cases.

I'm not based in China, bro. You're the one who went around accusing random people of being advanced persistent threat actors, and now you can't even keep your story straight.

Do not post blatant lies.

https://github.com/ducaale/xh is where it's at, haven't used curl in years.

Reply on news.ycombinator.com