Comment by chubot | Hacker News Reader

Comment by chubot | Hacker News Reader

Parent Full thread

Congrats to the fish team! Great writeup with lots of interesting detail.

I wonder if this is the biggest project that has moved from C++ entirely to Rust (or maybe even C to Rust?) It probably has useful lessons for other projects.

If I'm reading this right, it looks like fish was not released as a hybrid C++ / Rust program, with the autocxx-generated bindings. There was a release during that time, but it says "fish 3.7 remains a C++ program" [1]

It sounds like they could have released if they wanted to, but there was a last stage of testing that didn't happen until the end.

Some people didn't quite get the motivation for adding C++ features to Rust [2], and vice versa, to enable inter-op. But perhaps this is a good case study.

It would be nice if you could just write new Rust code in a C++ codebase, without writing/generating bindings, and then throwing them away, which is mentioned in this post.

---

Also the #1 gripe with Rust seems to be that it supports version detection, not feature detection.

But feature detection is better for distros, web browsers, and compilers:

Feature Detection Is Better than Version Detection - https://github.com/oils-for-unix/oils/wiki/Feature-Detection...

Version/name detection is why Chrome and IE pretend to be Mozilla, and why Clang pretends to be GCC. Feature detection (e.g. ./configure and eval() ) doesn't cause this problem!

[1] https://github.com/fish-shell/fish-shell/releases

[2] e.g. https://news.ycombinator.com/from?site=safecpp.org

To clarify, work on the rust rewrite started after 3.7.0, but the C++ code remained in a working branch on the git repo. Midway through the rewrite, we backported additions and improvements to fish scripts (most observable being new and improved completions) and a couple of important bugfixes from the rust-containing `master` branch to the C++ branch and released that as 3.7.1.

We never considered releasing anything with a hybrid codebase; aside from the philosophical purity of fully making the switch to rust, it would have been a complete distribution nightmare (we take package maintainer requirements very seriously). Moreover, the code itself was not in a very pretty state - the port was very much like trying to undo a knot: you had to make it much uglier in order to get it properly undone. There were proverbial tons of SLoC that were introduced only for transitional interop purposes that were later removed, this code was never held to the same quality standards (in terms of maintainability; it was still intended to be bug-free and required to pass all our unit and integration tests, however).

As mentioned in the article, we prefer to do feature detection when and where needed/possible. The old codebase was purely feature-detected via the CMake build system but we ended up writing our own feature detection crate for rust invoked via build.rs (maintained here [0]) though we just defer to libc on a lot (which doesn't do that yet). One side effect of the libc issue is that we're beholden to their minimum supported targets (though I'm not sure if that's strictly the case if we don't use the specific apis that cause that restriction?), which are higher than what we would have liked because we were fine with feature detecting and implementing using both older and newer apis where needed.

[0]: https://github.com/mqudsi/rsconf

> Feature Detection Is Better than Version Detection

The problem with feature detection (normally referred to as configuration probing), at least the way it's done in ./configure and similar, is that it relies on compiling and potentially linking (and sometimes even running, which doesn't work when cross-compiling) of a test program and then assuming that if compilation/linking fails, then the feature is not available.

But the compilation/linking can fail for a myriad of other reasons: misconfigured toolchain, bug in test, etc. For example, there were a bunch of recent threads on this website where both GCC and Clang stopped accepting certain invalid C constructs which in turn broke a bunch of ./configure tests. And "broke" doesn't mean you get an error, it means your build now thinks the latest Fedora and Ubuntu all of a sudden don't have strlen().

IMHO a broken toolchain is a broken toolchain and that's kind of outside the scope of autoconf -- and I say this despite having banged my head against the wall only too many times as a result of an odd toolchain misconfiguration leading me into chasing autoconf gremlins.

One thing about rust is that it has always treated cross-compiling as a first-class citizen. Cargo is very intentional about the difference between the HOST and TARGET triplets and you can't mix them up unless you are doing so intentionally.

The rsconf feature detection crate was similarly designed with cross-compilation in mind from the start and eschews running binaries in favor of some clever hacks to exfiltrate values during the cross-compilation process.

There is only one rsconf feature (retrieving compile-time constants) that is currently labeled caveat emptor as it does not support cross-compilation; perhaps I can nerdsnipe someone here into figuring out a workaround: https://github.com/mqudsi/rsconf/issues/3

I generally think autoconf etc should be defined to expect certain things by default (keyed by OS), and fail loudly rather than auto-disabling those features. If you really don't want those features, pass in --disable-foo.

I re-did Firefox's autoconf to do this back around 2010 (was contracting for Mozilla as a part-time job in college), after running into one too many features that were automatically disabled because of a missing library. There was at least one Firefox nightly that was missing an important feature because the build machine didn't have the required library.

Yes! Please fail if a feature tells me its on by default but it can't be enabled for whatever reason. Otherwise I need to hope that, among all the output of the configure script, I didn't miss anything about the script choosing to disable a feature.

The XZ utils supply chain attack also used this to sneakily disable Linux Landlock: https://news.ycombinator.com/item?id=39874404

Hm what's an example of those invalid C constructs? I'd be interested in seeing what happened

One answer is the __has_feature tests mentioned in a sibling comment. Then you are using a supported API, not arbitrary code. Browsers should probably support something like that, if they don't already.

But the arbitrary code is still a useful fallback, for when the platform itself doesn't support config probing

I think you're saying that "writing good ./configure is hard", which is absolutely true. But it's still true that feature detection is better than version detection.

Although Clang does set the `__GNUC__` macro and you have to distinguish it using the `__clang__` macro, Clang and GCC also both have very fine-grained feature detection features as well, both at the CLI level and in the preprocessor (such as the `__has_feature` family of builtins).

Reply on news.ycombinator.com