Feature flags in Bazel builds | Hacker News Reader

Feature flags in Bazel builds

pigweed.dev

45 pointskaycebasques22 comments

At what point do you bring in Bazel? I've heard some people say stuff like Python should always use Bazel, or that full stack webapps should use it, but it often just seems like extra cruft. Why not just have separate repos and use each language's tools?

For me, it's usually when I have more than three or four languages in a single codebase and there's codegen (e.g. protobuf, GraphQL, etc) involved. It's nice to be able to `bazel test //...` a whole repo, and building minimal, reproducible Docker images becomes comparatively straightforward.

That said, I've never made the leap in using Bazel for frontend stuff, I always use NPM/pnpm/yarn without any Bazel trimmings.

We had an integration test that involved many tool chains and languages to prepare. Rust compilation, WASM compilation, Docker builds shell scripts that produced a VM image, Python was in there somehow too, and then a test harness, written in Rust, that took all that to actually run the test.

With Bazel setup, and it was a beast to setup, developers could run all that, with remotely cached incremental builds, efficient parallel execution, and all reproducible (and same as CI) from a local developer environment with just one command.

I recently left Google and I do miss blaze. I'm working on some stuff that's rust and wasm and typescript and there are containers and I'm very tempted to use bazel for basically correct caching and stuff.

But even as a xoogler I think that the common wisdom of "don't use bazel until you can afford a team" or "use bazel if you are the only developer" are probably right.

My somewhat janky series of dockerfiles and bash scripts is at least debuggable to my business partner.

I'm just not ready to commit our future frontend developers to having to use bazel yet.

I'm sort of sticking to a lowest common denominator technology stack for most things except where we are spending our innovation tokens in wasm land.

But someday, probably. In the meantime I may setup sccache.

> Rust compilation, WASM compilation, Docker builds shell scripts that produced a VM image, Python was in there somehow too

was it through bazel rules? The worry is if some of those rules will get bug or missing necessary feature, it will be pita to deal with.

This is one reason why I’m quite interested in buck2; there are no built in rules at all, so if you need any changes, you can just make them.

Unfortunately the open source set of rules is a bit rough right now, with a decent happy path but a cliff if you’re off of it, but given some time…

The rules are open source. We did run into some road bumps in Rules Rust, that required patches or clever work arounds. But I believe the community has made great strides to improve Bazel’s Rust support in the last five years.

To be frank, I would avoid introduction of Bazel unless you are experiencing major pains with your current system and have at least a small centralized team willing to build expertise and provide support.

Bazel is definitely like kubernetes where you don't need it, until you need it.

If you got a big polyglot application its perfect for you. If your app is largely all in one language then you don't need that complexity in you life.

We leverage Bazel heavily across multiple very large projects where I work. It greatly simplifies dependencies across languages with different build steps. We also run Bazel in docker so everyone has the same build environment. Finally, we leverage Bazel on cloud build farms so very large builds don't happen locally and can leverage shared caches. This is really where the hermetic nature pays big dividends.

Our codebase uses a single build system for CPP, Python, CPP, GoLang, and more.

I'm spoiled, as I worked for 2-3 years at Google, and coming from gamedev - I was wtf is this thing, then when I quit I started missing it.

I've used with Java, Python, some C++, some R, bash, sawzall and few other langs. The beauty of it is that it can express dependencies across things from different languages (where this makes sense). Another cool thing (for Java) and FFI C/C++ code is that it can build custom java runner that embeds everything in single binary. So your FFI C/C++ code and the Java code becomes one binary.

But basically you can say - in order to build this, you also need to build this in a single language.

On top of that you can express tests too. And then it knew how to take/combine unit tests of same "size" and shard them to execute across machines.

And because you have to size your tests (e.g. RAM / CPU wise), it knows which to pick, and if your test does not fall anymore in that category it'll tell you to fix it. Might even offer command-line (nowadayws buildozer command-line) how to fix it. (That was ~2014-2016)

Then for C++ (and other langs) it'll track whether you've specified your header deps. To think about it, let's say you've said - here is a.cpp and a.cpp includes a.h - but forgot about adding explicitly a.h - Bazel (blaze) can catch that. One of the ways is simply to "symlink" only explicitly placed files in random temp folder, and compile from there - so "a.cpp" won't find "a.h" there and it'll tell - "Here run this command to add a.h to your target".

so actually quite the opposite - you want build system that unites all these different languages/runtimes/environments.

Not like - python/pip rust/cargo dotnet/nuget javascript/npm java/maven - because with each such different language specific build layer you lose how to express depedencies between them.

And not only that - in bazel deps are static, e.g. with a.h missing explicitly in your BUILD file, and bazel would tell you that. In the examples above is healthy mix of everything - from dynamic discovery to who knows what.

Then again, I get it - it's not the easiest to get through, the .bzl is dark magic sometimes, but the BUILD files should be easy to use to anyone - engineer, linguist, statistician, producers, tech art, etc.

> Not like - python/pip rust/cargo dotnet/nuget javascript/npm java/maven - because with each such different language specific build layer you lose how to express depedencies between them.

Its funny you highlight this as a feature, because my #1 piece of feedback for the bazel ecosystem is that it really sucks to have to give up language-native tools. Whether it's LSP servers, documentation that says "add this to your gradle file", "add this to your build.sbt", it's always an uphill battle.

And then it's an uphill battle convincing product engineers to forget everything they know, do it the bazel way instead.

I get why bazel is cool, but I think it's worth highlighting that for a lot of folks it very well can make their inner development loop worse than without it, particularly if they don't have someone (or several people) dedicated solely to developer experience.

Can you explain further? Yes, it takes more effort to express, or rather to create .bzl files to express these dependencies (e.g. let's say we have Python using C/C++) - but once it's done it gives a higher level (BUILD) language where this is easy to express... Now there are still many rough corners (especially when comes to dealing with non-mono repo deps), but I'd rather use this, than hodge-podge of shell scripts/makefiles/cmake/etc, where the steps are not even clear what to run first, and the confiendence if this going to be reproducible.

I am not really talking about cross-language dependencies. I mean simple things like Java product engineers who know gradle and want to use gradle and get grumpy when they don't know how to add dependencies to WORKSPACE. These people need to be trained to use different tools rather than use the skills they already know. That's fine, but it's definitely a cost to using bazel.

Or if you are trying to maintain bazel for an org. Invariably, you will receive requests of the form "I would like to achieve [well document thing in language-specific build tool], please help me do that in Bazel". The net result for me, who was once in that role, is that understanding the original build system, then bending bazel to match a certain behavior (usually with no or inadequate docs) is a huge amount of effort. Enough that I now am skeptical of the value-add of bazel.

For what it’s worth, gradle is probably the only other build tool that has good inter-language capabilities (and it is heavily used by the android toolchain). Most other build tools are specialists.

Large codebase in which you need a hermetic, repeatable, incremental build and test support. Note that in a hermetic build tests can be incremental too: if input and code do not change, there’s no need to rerun the test, and Bazel will skip it.

You'll get wildly different answers depending on who you ask. Most ex-Googlers appreciate Bazel. Folks used to ergonomic single-language builds dislike it.

In my experience, Bazel works best when:

- You have a monorepo.

- You have one engineer who knows it in and out and can stay current.

- You use several languages well-supported by Bazel (Protobuf, Go, Java, Kotlin).

The benefits are a single way to build and run all tests and binaries across all languages. Bazel CI providers (BuildBuddy, EngFlow) are much nicer than YAML engineering in other CI systems.

I have an old comment that's still representative of my experience: https://news.ycombinator.com/item?id=32831555

Not sure I fully understand Pigweed — is it basically an open-source reimplimentation of Google's developer experience for doing firmware development?

https://pigweed.dev/docs/overview.html

(If you're reading this, hello Keir!)

Reply on news.ycombinator.com