Hacker News Reader

Comment by Twirrim | Hacker News Reader

To ask the obvious unanswered question: Why would someone want to move to Bazel from Maven? What advantages does it bring?

View on HN

For me, the main advantage of Bazel is speed, when properly structured and configured, a Bazel build can:

* Perform incremental build a magnitude faster than Maven, zero changes build finishes immediately.

* Can have global caches for all builds across organization, so that a workstation build can automatically build incrementally upon a standard build, massively reduces build speed for big monorepo.

* Very fast, incremental tests execution: test results are cached, so only tests affected by the changed code are run. Test executions are parallel by default.

Some other things:

* Pull in dependencies directly from a git repository, without having to publishing it first like in maven, this can be a plus or a minus.

* Cross-languages build: you can have each of your module programmed in a different language, declare dependencies between them and use Bazel to perform incrementally build quickly and reliably.

That said, maintaining a Bazel build system is also a magnitude more complex than a Maven one, especially if you're not using one of Google main language: Java/C++. Plugins for other languages are of varying quality.

Given that Google has a Common Lisp codebase, I wonder...do they also have Bazel plugins for Common Lisp?

Yes

...and a lot of those arguments seem geared toward big codebases. Small micro-services, such as Spring Boot, with the bulk of the test execution going to @SpringBootTest tests, where parallelization has its drawbacks, may not profit from most of those "benefits".

Fun fact, it seems like Maven was also meant to service a variety of languages. There's even some support for polyglot POM files. Aside from the niche C++ or JavaScript ("frontend") plugin, that never did much.

Actually we have seen a ton of success using it for spring boot microservices - each microservice has its own BUILD file within the monorepo. Test execution in spring can be parallelized as needed (and not used if not needed!) but i find our tests run a lot faster under bazel due to the caching anyway

Can we agree that "it depends"? If I have a resource-starved build runner (looking at you, there, Bitbucket Pipelines), and I have a good handful of parallel integration tests, each spinning up a few Docker containers worth of dependencies, I will end up troubleshooting out of memory situations. I also don't think you get to leverage application context caching, for parallel test execution, and if you did, I'd be concerned about unpredictable conflicting state between those integration tests. If I can already run my tests for a single microservice in about two minutes, the parallelization just isn't worth it to me (in CI). And, oh, I'm not in a monorepo.

Is test result caching only possible due to a fundamental design of Bazel, or could you modify for example the maven surefire plugin to achieve the same?

> Perform incremental build a magnitude faster than Maven, zero changes build finishes immediately.

I see benefits of that in projects that build hours, but in every one of my maven projects (non-hobby) I always do clean before package/verify just to make sure the project builds.

``` mvn clean verify ```

And I'm certain it will work in CI also.

People tend to run ```mvn clean verify``` because Maven caching is unreliable, and lead to build failures.

However, Bazel cache is a lot more robust, so you almost never do clean build. The only time I had to perform a clean build was when I accidentally upgraded my C++ compiler and broke the cache.

Bazel guarantees that each build step depends only on its declared inputs, and that each step is deterministic. So if the inputs to a step have not changed, Bazel can skip that step and use a cached copy of the result of that step. If you break up a large multi module build into a tree of small steps, Bazel can perform an incremental build after a small change in a few minutes, even though running the entire build from scratch would take hours.

> Bazel guarantees that each build step depends only on its declared inputs

How does it enforce that? Does it sandbox the compiler or something? Java compiler annotation processors can perform arbitrary IO.

Exactly that: https://docs.bazel.build/versions/master/sandboxing.html

There's also (I believe experimental) support for using Docker as a sandbox backend. That ends up being useful if you're using the remote build execution support: you can run a build locally in exactly the same environment it will run on a build farm.

I wonder if this stops you reading the clock and random number sources during build?

(I once accidentally baked a time and random number into a binary myself - and the random number thing would have been a serious security bug if we had not found it with test coverage. Would have been better if the build system disallowed it.)

You can actually input system stuff like this via: https://docs.bazel.build/versions/master/user-manual.html#fl...

They already have support for build times and labels, but you can add arbitrary stuff like got hashes too

It can't stop you doing dumb things in code (e.g. use #pragmas or variables in C++ code) which will probably make your build behave funny, because it'll cache outputs not knowing that they're supposed to change.

You can of course define these as variables, but that'll just drop all your caches on each build which eliminates the main selling point of Bazel.

I don’t see Windows in supported systems for sandboxfs. I guess that would make bazel not suitable for most developers.

You have to declare all input explicitly, including your compiler and Bazel will monitor them: Any changes in input files, compilers or even some environmental variables like $PATH will trigger a rebuild.

Bazel can use sandbox as an additional layer to prevent you from accessing undeclared inputs, but the sandbox is optional and can be turned off.

Maven does exactly that as well. The term is "artifacts" in Maven.

Most plugins do not though, and will happily do stuff like run static analysis or lint rules over code that hasn’t changed from the last build 30 seconds ago.

To me, the strongest motivation for Bazel arrives with a complex (slow) build spanning multiple technology ecosystems. Bazel does a great job building all of them, without privileging one over all the others. It has all the key features you'd expect, dependency graph control, caching, distributed computation, etc. You can add support for whatever local complexities invariably arise with custom rules.

The Java ecosystem has much of that available also though - even a large Java-only project is somewhat tough to justify moving away from the tools almost every developer will arrive already knowing.

It came from Googlers, who are kind of notorious for not using the rest of the industry’s tools. Sometimes it’s because they pioneered the problem space, and their own tools really are that much better. Sometimes their tools used to be better but are now just tech debt. Sometimes it’s promo-driven development.

> Sometimes their tools used to be better but are now just tech debt

This is how I feel about most of Googles Java libs (guice, guava, gson). They used to be better, but now they're just tech debt.

What are the best-in-class replacements for those?

Guava -> The JDK Std library, unless you need more performant collections provided by it.

Guice -> If you need to support the jakarta.* namespace the only real answer is Weld or another CDI compliant implementation. Otherwise guice is probably still fine.

GSON -> Jackson as always.

Also, how does Bazel compare to Gradle?

JetBrains and it seems basically every library has support for Gradle and sometimes Maven, but not Bazel. Gradle is verbose and has a really steep learning curve, but it's battle-tested and does its job really well. I don't really see why anyone would use Bazel instead.

Gradle’s remote build cache which caches at the task level is nice and granular. Also we’ve never had a problem with a change requiring a rebuild not being detected.

One thing that Bazel has that I wish gradle did was remote compilation and remote test execution rather than just remote caching of artifacts.

For the most part, I think you've got it.

A gradle build can be very complicated and very long.

Bazel is relatively easy to learn and relatively concise (again that's relative to Gradle which for a Java dev basically requires you to learn 1-2 new languages if you want to understand the build file syntax), and IMO is MUCH better if you are working with multiple languages.

However I haven't tried to use Bazel with things like native libs in the JVM... so it's possible that my Gradle opinion is biased because I used it with more complicated use cases.

I think they're fundamentally opposed in their philosophy.

Gradle: Just slap any kind of code into our build files and we'll run it. Everything is code, everything is customizable and you can muck around with everything.

Bazel: Here are a few of build rules. All compilation needs to be deterministic, all tasks need clear deterministic outputs and all task generate clear deterministic outputs. It's very rigid and it uses that to get significant performance wins, especially when using a build farm.

That’s absolutely not what Gradle is. Gradle only provides a DSL that is supposed to be used to describe the build process deterministically, and add some configurable scripting on top.

For anything more complex, you should write plugins.

Nonetheless, I would be interested in seeing a detailed comparison of the two. As far as I know, gradle is quite capable once someone understands what he/she does.

https://melix.github.io/blog/2021/01/the-problem-with-gradle...

The Bazel plugin for IntelliJ works rather well - I’ve used it for Python and Go quite extensively and it “just works”. I assume this is the case for Java too (though have not verified). The main issue is that releases often lag IntelliJ releases by a not-insignificant amount since they’re (I think) done by Google and not JetBrains.

Correct. That plugin is developed by Google primarily as internal dev tooling, and the version of IntelliJ supported internally is generally not bleeding edge.

You get hermetic and cacheable builds, but only if you’re willing to abandon the whole Maven plugin ecosystem and handle a lot of codegen yourself.

Bazel does embrace code gen, but to be honest I've found rules for most of the code gen I've needed to do and they plug in nicely.

I still don't understand, Maven artifacts are hermetic and cacheable, no?

* Bazel offer much more granular caching, because a Bazel BUILD target is often much smaller than a Maven artifact. For example: a Java BUILD target is often just a single package.

* SNAPSHOT maven artifact are not hermetic and not reliable for caching: Maven only re-fetch SNAPSHOT once a day by defaults, so you will miss changes if you do not explicitly update it. In Bazel, any changes in a BUILD target's input are detected immediately and will trigger a rebuild.

Bazel rules are supposed to only read specified inputs and write specified deterministic outputs, and they’re skipped if nothing changed. Maven plugins can do whatever they want, and you decide whether to publish what's on disk.

Distributed builds, and even distributed cache of build outputs. At google scale a distributed cache means you don't even need to build anything except the smallest code unit that changed--your CI systems, etc. have already built and warmed the cache for almost any other thing you want to build.

I thought that the distributed part wasn't opensourced.

For the cache: https://docs.bazel.build/versions/master/remote-caching.html and https://github.com/buchgr/bazel-remote You can actually just set it up to use a shared S3 bucket to hold the cache.

For builds there are a few options to check out: https://github.com/jin/awesome-bazel#remote-caching-and-exec...

That all looks like a massive security vulnerability. Anyone with write access to the cache can upload a malicious build artifact and end up getting it compiled into everyone else's binaries, effectively giving them access to act as any developer or the ability to get their code sneaked into a release undetected. The page doesn't describe that risk at all!

I read what you meant, not what you said, had the same question.

Reply on news.ycombinator.com