Maybe you have some really complicated use case that makes cargo insufficient, but really, it's pretty damn nice and it's integrated with everything.
There are languages that need bazel, rust isn't one of them
In our monorepo, we ended up just using a Makefile, since it basically does that. Of course the Makefile is now a few hundred lines and we have about 60 *.compose.yml files, but that's another story.
Our current solution still uses the Makefile, but we basically split the monorepo into a Python part and a TypeScript part, and each of those has its own monorepo build system (Yarn workspaces for TS, and Poetry with Pants for Python).
Other big complaints were:
- big CPU hog
- bad IDE integration (esp scala/java for intelliJ)
- it lets people write i-cant-believe-its-not-python, which is far too much rope to hang yourself with. I've seen horrors.
Yeah, that's an effective way to sell someone's promotion, but it's always a promise that is never met and instead it's a constant source of problems.
Sometimes reinventing the wheel just buys you a wobbly, squeaky wheel.
It's just, when I joined Google I found a build system I didn't hate. This was the first time. I don't actually love Bazel (Blaze internally), but I love that I don't hate it.
Every time I have to learn a new build system in open source work I groan. Why are there so many systems for essentially the same thing?
There is definitely a need for better (and Meson is the foss answer, as well as Shake). But Bazel need a context that OpenSource do not have. Time and a clean slate of dependencies cooperating.
Also noone fund tools for DX for OpenSource devs :) only for JS and even that...
But the open source world has the resources to write new build systems (and ecosystems!) time and time again. This argument does not hold up.
What leads you to believe that writing new build systems is a popular hobby?
I mean, cmake is the de facto standard for C++ projects and it has been for what? Over a decade? What does this tell you?
It’s not solely the user’s responsibility to hold it right. But Bazel is and has been developed in the “open source context” for long enough now that your argument doesn’t hold up imo.
There are some new talks coming out external from the Bazel maintainers which do a better job of this, but you still have to hunt for them.
This is made worse because each of the rules packages, which are super useful, have their own idiosyncrasies which only make sense with a solid foundation in what problems Bazel solves, how it works and how to write/read rules.
My other wishlist for Bazel was an easier time setting up a hermetic sandbox without having caveats. This is improving, so I am optimistic on this front.
For context, I am not a Google (or ex-Google) engineer and I’ve had to work with a ton of different build systems on different levels of the stack.
We had a googler at our company switch everything to bazel and it just slowed everyone down.
There really is something about Google engineers, this sort of subtle arrogance that the Google way is superior to everything else is existence. Maybe I'm the ass hole but I wonder if anyone can relate?
The issue is that to make it really nice:
- you have to convert everything to it, not just almost everything
- you need to set up build servers and lightning fast caching servers
- you need a team dedicated to building bazel rules for whatever doesn't already have rules (see: you must convert everything). This team is a bottleneck
Regular google engineers aren't exposed to these issues, because inside google it just works. It's only once they try to deploy it themselves that they realize it's a sisyphean task.
The thing is, bazel actually is really good at solving the issues it was designed for, it's just most companies do not have those particular problems acutely enough that it's worth paying the cost to switch to bazel.
This is a major issue. A good build tool shouldn't require a dedicated team.
Ideally a build tool should function as sort of a settings menu for the project.
Build systems are complex because they solve issues which only exist because of how OSes handle libraries and executables. I am pessimistic we can ever solve that issue in a capitalist society. I will be optimistic if we have a new major OS that isn't Windows, Linux, or MacOS in our lifetime or we solve death.
I'd love to hear if anyone can make a case for Bazel knowing that cmake+build cache tools like ccache buy far better performance improvements than any full or incremental build.
When I use Bazel it is because:
- I want to build nearly everything from source in a controlled hermetic environment. No more "well it works on my machine"... only to learn the developer's libs leaked into the build environment.
- I want to do this with a single build system that works across many languages, and to build libraries which are used in many different languages.
- I want to do this within a sandboxed environment for peace of mind when compiling thousands of dependencies.
- I am willing to pay the money and time to setup remote executors to keep performance acceptable.
It's just that I really really hate maintaining Makefiles and CMake configs.
Admittedly in the latter case it's because I have never grokked CMake, but also... I don't want to grok CMake, I grok a perfectly good way of building C++ already. And it can also build Go and Rust and Typescript and Proto and JSonnet and it can run my janky codegen shell scripts and if we migrate to Carbon it will build that too and I won't even have to read the documentation.
Not saying I think everyone should migrate to Bazel (I hope I will not become That Ex-Googler), just explaining where the urge comes from.
CMake is a high level makefile/build system generator.
There is nothing to maintain in modern cmake. You just set what executables/libraries you want to build, set their dependencies, and you're done.
And of all alternatives you chose to push, you pick Bazel of all things? Feature-incomplete and requiring all sorts of low-level maintenance?
The logic is convoluted, documentation is over complicated and the language contains too many primitive functions and ways to shoot yourself in the foot.
There's really nothing good that's available.
Only lcov is supported, and due to the sandboxing it's impossible to get other tools working since they always spit out extra instrumentation data that's neither cached nor available on the next sandbox.
Does blaze support code coverage properly? Does Google just not care about code coverage? Am I just missing some obvious thing in the documentation?
The most useful leads I've found so far are:
* The design doc for the (abandoned?) coverage framework [1]
* A prototype of GCOVR support [2]
* Comment on getting llvm-profdata working [3]
[1] https://docs.google.com/document/d/1-ZWHF-Q-qCKf19ik-t33ie58...
[2] https://github.com/ulfjack/bazel/commit/e9f21bbf562c1f6006eb...
[3] https://github.com/bazelbuild/bazel/issues/8178#issuecomment...
I forget the exact details but I think you can do this with a custom run_under test wrapper used during coverage. The wrapper runs your test, with whatever custom coverage system you have, then afterwards it converts to LCOV. Sometimes this might require updates to language specific rules because the test runner itself needs to be configured.
The bottom line is that Bazel upstream does not put a lot of attention into this, so it’s under developed outside of Google, because Google internal does not use the OSS rules.
The testing is not all you need to modify. You need to modify every compile and link step, since you get byproducts of the instrumentation on every step.
>Sometimes this might require updates to language specific rules because the test runner itself needs to be configured.
That's what I found, too, however I feel like rewriting the C++ rules is a huge undertaking, since they're not defined in Starlark.
I have to call bullshit on this. With C or C++ you need to build the whole project with specific compiler flags to be able to generate code coverage reports. This is not something you implement by tweaking a setting in unit tests.
Code coverage is pretty well supported in the internal ecosystem but I would not be surprised to hear this is because we designed Bazel around the system Google was already using for coverage.
What you're actually saying is that internally Google managed to get a single and very specific use case to work and forced all projects to either adopt it or have no code coverage.
Why?
The world has been running on make for decades, and it runs just fine. Meanwhile, does Bazel support basic stuff like shared libs or vended libs that are not deployed system-wide? Last time I checked, it didn't.
Can you actually provide a working example?
Tools commonly available in distros are typically much easier to use for outsiders.
Definitely a ton of rough edges on Bazel but the passage of time is rounding them, even if progress uneven and frustratingly slow.
> Tools commonly available in distros are typically much easier to use for outsiders.
Which part about autotools or ninja is easier?
I'd love to hear what leads you to believe that adding yet another build system in the form of Bazel would solve that problem for you.
Though I seriously wish there were a better option, because there are a _lot_ of usability things with Bazel that make me very frustrated when working with a repo that is "merely" quite large.
Do you think that the problems with cargo mentioned in TFA are exaggerated or solvable? I think Bazel's support for caching, parallelism and custom rules are real advantages for large projects. What's wrong with this argument wrt Rust/cargo?
I'd say they're exaggerated at least slightly, yes.
It starts out by saying "Cargo isn't a build system" (which is already the sort of provocative exaggeration designed to increase participation), and then immediately veers off and starts talking about running tests. Cargo is a build system and dependency manager. It also happens to provide `cargo test` for running unit tests and integration tests, but this support is relatively simple, and if you need to do something weird then you're welcome to write a script to serve as your test runner. But a build system is not a test runner; a build system produces artifacts, and a test runner runs tests. The author appears to be annoyed that Cargo is not a build system for web assembly, which is part of their test process, and obviously Cargo is not. This is a somewhat eye-rolling critique, and does little to motivate rewriting the world in Bazel.
The second complaint is that Cargo uses file timestamps to decide when to recompile files. I suppose it is possible that Cargo could then proceed to lex any changed file to produce a comment-free token stream, hash the token stream, and then compare that hash to a previously-computed hash that you've stored in a database somewhere. Frankly, this sounds like a lot of work for essentially zero benefit; how often, exactly, do you go into a file and change only a comment? This is a weak justification.
It then mentions how changing git branches can invalidate the cache, but I don't think that's true. Git only updates timestamps on files that have changed when changing branches. If a file has changed when changing a branch, that is, again, probably because it has actual changes, and will almost never be because only a comment has changed.
It then makes some suggestions about Cargo not properly tracking changes to some global system resource that they have, which is simply too vague to comment on, and the fact that their prior reasons have been so weak makes me think that if this, of all things, is what they're trying to be vague about, then I assume this must be so unique and specific that even they recognize it's a weak critique.
Speak for yourself. Our Rust monorepo has nearly a dozen apps and counting, lots of shared libraries, C++ dependencies, etc. Tests require multiple cargo invocations.
We need a proper build system with distributed build caching. Cargo takes forever and doesn't understand the workloads.
I agree that Cargo is already near perfect and I wouldn't suggest you switch to Bazel if you have a Rust project.
In our case we have a C++ project with some Python, some Rust. Having everything in Bazel here is good for us.
It's a good thing then that cmake+ccache work out of the box by passing the caching tool of your choice like ccache to the compiler launchers.
I feel like our use case at Grapl wasn't particularly crazy. It was just slightly more than "I'm a dev building a library and publishing it" - we had a workspace with multiple devs and CI/CD. CI/CD was really slow - issues with caching, rerunning unnecessary tests, etc.
Plus we had some Python code so unifying our builds would have been ncie.
To add onto this, Grapl had a moderate sized codebase with less than 10 engineers. I do think we could have done some optimizations in cargo, but it's simply much easier to use a tool like Bazel/pants etc that does the work for you as long as it supports your language of choice.
What problems, exactly? The author acknowledged cargo works but hand-waves over vague claims of "it does not track dependencies well or support arbitrary build graphs", followed by acknowledging those issues are either non-issues or solved problems whose solution was dismissed without any good reason (caching tools).
The author's claims boil down to "I wanted to shoehorn Bazel but had no good reason to, so I double-down on Bazel's only selling point while ignoring everything".
It's ok if anyone just wants to try stuff without having any rational or coherent argument to justify it, but let's not call that "problems".
Nonsense. All build systems implement a DAG of build targets.
> why caching tools can't be used effectively
Nonsense. There are a myriad of caching tools freely available, from ccache, sccache, build cache, etc. To get them to work, all you need to do is pass compiler launchers to them, which means setting a single env/build flag.
The author is either trying hard to make a mountain out of irrelevant molehills, or is outright being disingenuous.
Bazel is being sold as a solution desperately grasping at straw-like problems, while being feature incomplete.
And the example they gave is one that doesn't work so well.
> ccache, sccache, build cache, etc
The example they gave included the build cache misfiring. ccache doesn't work with Rust. sccache does, but can't be used with incremental compilation or when building binaries, making it essentially useless for what they needed, i.e. speeding up CI and local dev.
I really have no idea why you're still avoiding reading the article you're dismissing.
Bazel is a nightmare. I wouldn't wish it on my worst enemy.
Don't look at headcount to gauge complexity, difficulty, etc.
What I've experienced is the larger the team the more likely to have dilution of ownership. The less ownership the less likely each person will spend the energy or fully understand solving root causes.
In two ways, one being that more people legit need a more complicated system. You want that so that they interact with the system, not with each other.
The other being a bit of a Parkinson's Law. That is specifically that you will expand work to fit the allocated time; but I posit that you will also expand work to fit the people doing it. So, more people pushing ideas into the codebase will keep more ideas in the codebase. Even if fewer would work.
What parts of Bazel were a nightmare? What problems were fixed by these 4-5 people that were previously impacting all 25 people routinely?
It still doesn't support post build steps, and integrates poorly with other tools that are used for more general builds.
Hey look another nimwit that make stupid assumptions without understanding the nature of the problem being solved.
> and it's integrated with everything.
Oh wow Cargo integrates with Python, shell scripts, C++ and node JS? Holly sh*t that’s amazing.
/s