Check out Bazel for Rust.
It allows:
* caching of artifacts.
* shareable caches between {developers, build jobs} based on hashes.
* remote distributed builds (on very many cores).
Something else is that most projects tend to build everything from source, even protobuf dependencies, so it takes me an hour to get the initial build of envoy done.
I do write templates but not very often. Also when I do I try to consume those in a single file so that they could be declared there as well without propagating into a header. Not a good idea for writing libraries but works for applications.
>"Chromium has 12 million lines of C++."
I am far short of that of course.
if you're lucky enough to have enough RAM to spare (which would be, like, a lot!) you can also build in a tmpfs ramdisk!
it can also be very memory consuming so as soon as you hit swap things slow down a lot.
Any example?
Not nearly as flexible or powerful as Bazel, but also vastly simpler to setup if all you want is caching.
But for the "any project" use case, the combination of Rust's compilation model and how crates.io works makes this hard. Due to Rust's compilation model, if crate A depends on crate B, then the version of crate B it's been compiled for can't be changed: you have to use that version. It also means that the features of the crate that have been enabled can't be changed. "Normal" Rust projects can quickly get into hundreds of crates.io dependencies in their DAG. Those are the very dependencies you want to use ssccache for. However, due to how crates.io works, basically every day you get a new updated crate in your DAG, and this invalidates all the crates that depend on it, including your own.
In a single project use case, this is no problem because everyone uses the same Cargo.lock so they use the same dependencies and the same features of those dependencies. But if you don't share a Cargo.lock, the problem is too wide.
There are some solutions like freezing crates.io according to a schedule and only pushing out updates once per week or so. But this still leaves the feature issue unsolved. Plus it is probably against the idea of crates.io to have updates available immediately. Another solution would be API only dependencies, but this is harder to pull off.
This is not a complaint about Bazel specifically, its fantastic, and easily my favourite build system bar none.
However it cannot cross compile Rust. This means if I’m developing on my MacBook, and I want to compile a Rust binary and put it in an Ubuntu docker container, I can’t do it on my host machine. I need to copy the source into the container and build it there, using multistage builds.
But this is -extremely slow- because it cannot take advantage of Rusts build caching. I’m talking 10-15 minutes for my small Rust project.
Has anyone run into this? How do you work around it?
I’ve considered running a Bazel remote execution server on a local Ubuntu VM, but this feels like so much extra complexity just to use Rust, Bazel and containers.
Apple ld doesn't support Linux as an output target, so you need to use GNU ld or LLVM lld instead.
Code examples at https://john-millikin.com/notes-on-cross-compiling-rust#baze...
There are two common ways i'm familiar with. Cargo Chef and Manually. Manually can be a bit involved, but you basically just build with empty source and prune your fake-lib files from the cache dir. Cargo Chef is very easy.
My docker images build as fast as my local images. Though if you have many crates in a workspace it can start to become difficult.
I've been using them locally of late, and they're excellent for storing state that's not obviously identical across invocations of the same layer.
We achieved textmode supremacy, and jart achieved binary-portability supremacy.
There seems to be a pattern here :)
The problem is that Google is well-known for unceremoniously dumping stuff.
So my first question about any Google tool is: "Does it have enough non-Google people supporting it?" because, if it doesn't, it will die as soon as it becomes a political football.
My second question about any Google tool is: "Are there small projects using it and singing its praises?" Can I use the tool to do what I need to a handful of files after reading a couple of web pages for 15 minutes and have it work. I don't need to scale to 4 gazillion foobars--I do need to get stuff done without an IT staff of 100 people. From a cursory look, Bazel fails that question.
My final question about any tool is more personal: "Does it use C++?" because, if it does, then integrating with anything other than C++ is going to be a gigantic PITA. At this point there are plenty of fine alternatives to C++ that don't suck. If you don't have a clean, pleasant way of talking to things that only understand C library conventions, you fail and I'm moving on.
It adds yet another indirection layer on what those runtimes expect, the IDEs aren't prepared for mixed language debugging with anything else, and their build systems also need some additional nurturing.
I think you've got that the wrong way around; if you can talk to things using the C library, you automatically get reusability from Java, C#, Python, various Lisps, Ruby, Rust (Go?), and almost every halfway popular language in existence.
What alternative do you know off that is more widely supported by all programming languages?
I'm in full agreement with the GP, I just misunderstood the text (that's all on me, he was very clear).
Vulkan development, for instance, drives me up a tree. Vulkan has probably the best C API I've probably ever seen.
Vulkan support libraries like the Vulkan Memory Allocator all tend to be C++ because that's what GameDev tends to use. And they're an absolute nightmare to integrate with anything not C++.
I mean, even the C++ guys eventually gave up. Look at the proliferation of "Header-only" C++ libraries because integrating with the C++ compiler and linker is simply too painful.
Per the article: bazel + rules_rust should have the flexibility to override the linker flags that Cargo may take as required since that would be a property of the bazel toolchain used.
It's a nice amalgamation of how cargo works and how bazel works.
In general bazel supports hermetic builds, multiple toolchains, cross complilation, and ways to compile multi-language projects.
I still wish that Cargo.toml didn't support build.rs as it can cause a lot of system-dependent problems that bazel sidesteps entirely by being hermetic.
I need this in my life.
Empathy is essential.
I see the linked resources are to docs only.
If it's any help, here's a bazelized Rust demo implementation:
https://github.com/mihaigalos/code_templates/tree/master/rus...
[1] https://developers.facebook.com/blog/post/2021/07/01/future-...
This sounds like something that nix is optimised for. The inputs into building each package is captured so having different feature flags would just create different artifacts.
Reading an issue I've opened almost 2y ago [1], seems the backend requires the client to have a specific gcc version.
That's a strong limitation imho.
[1] https://github.com/bazelbuild/bazel-buildfarm/issues/545