Hacker News Reader

Comment by reissbaker | Hacker News Reader

In my experience, Bazel is a net negative for most teams. Pretty much every JS engineer is familiar with npm; a tiny fraction are familiar with Bazel. Ditto with pip, cargo, etc. And it doesn't solve the hard part of the build process, which is distributed builds. Most of the user-perceptible value of Blaze is making builds fast by farming them out to a zillion machines — that's why it's called "blaze," because it's fast! — and Bazel doesn't do that for you.

And it's clunky, and you need to teach every new hire how to use it. The juice just isn't worth the squeeze. Just write an adapter in your build infra for the well known tools and be done with it. You'll get much more value putting work into something else, like code review tools, testing, dev environments, staging...

View on HN

Being familiar with npm doesn't save the time it loses you on its clunkiness. I see for myself and other engineers around me that we waste real time every week clearing our yarn caches, waiting for "yarn install", realizing we forgot to run "run install" after syncing and we have to re-merge and re-upload, etc.

Here's a real-word example from last week: I spent several hours doing creative hackery trying to convince a particular compiler to compile the same file using different dependencies depending on context. That would have been a few minutes of work with bazel, but when your build process consists of "run this off the shelf compiler" and the compiler doesn't have any native support for building two different versions of the same thing with slightly different dependencies then you're in trouble.

Teaching new hires bazel is a one-time cost, and the only thing they'll need to get started on their first day is "bazel build <target>" and "bazel test <target>". When you don't have bazel, on your first day every new hire has to read through a gigantic wiki page explaining how to set up their dev environment (with different sections for mac and linux and special asides describing what to do if you're on a slightly outdated version of the OS, and then the wiki gets out of date and you have new engineers and the infra team all wasting their time debugging why the instructions suddenly stopped working for people on slightly newer machines, etc.)

There are many remote build backends for bazel available - some open source and some proprietary (EngFlow and Google’s RBE). Even without a build farm it will still be a huge performance boost due to caching - something make and co (recommended in sibling thread) can not do along with a bunch of other stuff that bazel does for you.

I've heard the "huge performance boost" argument before, and in my experience it's often kinda marginal, or even negative because the native build tools have been optimized for their specific task. (Sure, I'll give you that it's better than Make, but is it faster than the Go compiler?)

I wasn't familiar with EngFlow; it looks like it's a startup that raised seed funding less than a year ago. I think what you're referencing with "Google RBE" is a Google Cloud project now rebranded to "Cloud Builds" — which supports the underlying native tools like npm, pip, etc without requiring you to switch to Bazel.

Bazel is better than Make for building large C/C++ projects (although it's hardly the only game in town for "better than Make"). But aside from that use case, in my experience it's not really worth the hassle. You can get most of the benefits you want without using it, and people are already going to be familiar with the tools native to the ecosystems they work in like pip, npm, etc.

> (Sure, I'll give you that it's better than Make, but is it faster than the Go compiler?)

It depends on what you're doing. If you're compiling pure go code that's truly only go, no go build will be faster. But if you have cgo or generated code or generated data or config files or...then well maybe you want something that is more flexible. And of course if you aren't building just go, then things get complicated fast. What if you have a backend and a frontend? `go test` probably isn't running your webdriver tests that compile your typescript somewhere in the build pipeline. Having a unified toolchain with one command (`blaze test //...`) is valuable compared to various test.sh (or make layered on top of n independent build systems or...)

And of course if you're like me and need to do things that involve reasoning or searching about code and dependencies, blaze is super necessary. "Find all of the dependencies of this file and test only those" is a question that most build tools aren't even remotely equipped to answer.

So in polyglot situations, bazel and similar prevail I think, but there's absolutely a point below which you don't care (and that point is going to be basically only hit by applications, not libraries).

Depends on which Make we are talking about, commercial Make tooling like ClearMake certainly can do it, since the late 1990's.

I actually worked with clearmake and bunch of other rational tooling in one of my early gigs and don’t remember much in a way of cache improvements. Soon as you’re doing some non-sandboxes io which make-based tools totally allow it’s out the window anyway

All object files and libraries would be cached and shared across builds using the same views (Derived object sharing).

Curious how clearcase works I found https://www.ibm.com/docs/en/rational-clearcase/7.1.2?topic=s.... They appear to be hijacking the open() and write() syscalls, so in some way they actually do have a sandbox that provides more accurate knowledge about the build graph than what the makefiles tell! Otherwise yes, makefiles themselves are very unsafe and frequently have errors such as underspecified dependencies or implicitly produced outputs, with no guardrails that can prevent them.

Wether or not that sandbox blocks incorrectly declared dependencies is unclear. Last time I used clearcase many eons ago it surely did not. Our project had tons of classic makefiles issues like not depending on included headers. Remote builds were also magnitudes slower than local builds, our network was maybe not the best, also reading the page above how the “shopping” algorithm works you can imagine it being fairly slow anyway. Maybe that was best, imagining how incorrect dependencies mixed with remote caching would result gives me nightmares.

Reply on news.ycombinator.com