Hacker News Reader

Comment by simplotek | Hacker News Reader

From the announcement:

> One of Bazel's most powerful features is its ability to use remote caching and remote execution.

What's the value proposition of Bazel when build systems like cmake support this out of the box with third-party tools like distcc and ccache?

View on HN

(Opinions are my own)

I think that this is a problem with branding which unfortunately Bazel is using. I think remote caching and remote exec are not the best features that Bazel has to offer.

First to answer your question: distcc and ccache are great but they offer a limited amount of flexibility in what can be distributed. Bazel's approach is generalized and operates on this basic set of steps:

1. Configure a "target" (define inputs and outputs) 2. Load the deps for the target into a sandbox. 3. Run a command in that sandbox. 4. Copy the outputs out of the sandbox.

With this approach you can execute anything as it's just running a shell program with some args. For example: lets say you are building a game and you have a process where you take raw models from your designers and compress them into an efficient storage format. This can be a `genrule` in your BUILD file and instantly get rerun whenever anyone submits a new change to the models. Now that it's been run in CI all of your devs will not need to rebuild this on their machines. That's the goal. Basically: it is more generic than caching a specific language's data. You can cache everything.

My favorite part of Bazel is the layer of abstraction. Rather than thinking of build steps, what args to gcc are getting passed, etc you are thinking about libraries, binaries, tests, and other artifacts.

Also, this abstraction of inputs and outputs makes it so it's easy to eventually build higher level abstractions. For example, this should be possible at some point:

    cc_binary(
      name = "foobar",
      srcs = [...],
      hdrs = ["foobar.h"]
    )

    # Automatically generates jni binding
    # for a C++ library just by reading the
    # header file and binding all types.
    java_cc_binding(
      name = "foobar_java",
      deps = [":foobar"],
      namespace = "::foobar", 
      generate_class = "foobar.FoobarBinding",
    )

This doesn't yet exist but it could and it would be pretty amazing. Or, another example, imagine AWS publishing a `lambda_binary` rule which you could pass any `*_binary` and run your binary into which automatically generates a packaged and ready-to-go artifact which you could upload to AWS.

I think the framework to allow others to build on this abstraction and make things take less effort overall is the huge value add.

None, bazel's caching implementation is broken because they don't even know or specify what constituents a build hash/key. See this issue from 2018 that's still open [1].

[1] https://github.com/bazelbuild/bazel/issues/4558

Probably because building targets with tools outside of the workspace is an antipattern, as it violates hermiticity principles. In fact, Bazel generally makes it quite hard to do this, so anyone who ends up in this scenario must have jumped through many hoops to get there.

I agree that the linked issue is legitimate, but I'd argue that this isn't a problem Bazel itself needs to solve--you should fix your build to be fully hermetic.

Non-Hermetic is the default for C/C++. And if you plan on using system provided libraries to support multi OSes then you can't use it.

That is precisely the point--using system-provided libraries in your Bazel project is an antipattern that should be avoided.

It's literally the default. How can the default be an anti-pattern. I doubt you're using C/C++ because you don't seem to understand the issue.

Learn to take others seriously without asking for credentials. Whether or not I use C/C++ is irrelevant. Bazel is flexible, and you can use it however you want, correctly or incorrectly. The principles I've mentioned above are language-agnostic, and are recommended best practice regardless of whatever programming language you are building.

https://bazel.build/basics/hermeticity

> When given the same input source code and product configuration, a hermetic build system always returns the same output by isolating the build from changes to the host system.

> In order to isolate the build, hermetic builds are insensitive to libraries and other software installed on the local or remote host machine. They depend on specific versions of build tools, such as compilers, and dependencies, such as libraries. This makes the build process self-contained as it doesn't rely on services external to the build environment.

My overarching point is, you should fix your build rather than point blame to the tool's authors because you're using it in an unsupported way.

Many people believe that the "traditional" way of building C/C++ applications is an antipattern. Such a belief is, in fact, a core reason to adopt bazel. If you don't believe that, then bazel may not be for you. It is intentionally opinionated in a way that you aren't.

I'm assuming you're referring to the golang model of statically linking everything. That's not really doable when many popular libraries are (L)GPL'd like glibc and libstdc++. It also doesn't work if you want to provide a shared library and need to be compatible with every possible system. That's not my opinion it's just a deficiency of bazel.

Then you build the lowest-supported-version of GCC and glibc, use that as your toolchain in Bazel, and build a dynamic shared library as normal. Using a system-provided toolchain also works, but you have to build on that system using something like Docker, which is certainly an alternative to Bazel but isn't quite meant to serve the same niche.

Non-hermetic is non-reproducible. I can only produce the same build outputs as you if I use the same toolchain, which essentially implies I must run the same OS and patch level, among a host of smaller impurities that can change the output of a build.

Sometimes this is desirable; for example, if you are packaging for a Linux distribution. But that's not the use case Bazel was invented to serve.

Not that I'd recommend it, but if you symlink your system library into the bazel build area, as long as your sandboxing setup don't hose you (or you just turn it off), bazel will track system tools/library in the same way as everything else.

Bazel's rules_cc even has a system_library.bzl you can import a `system_library` from that automates this for you. https://github.com/bazelbuild/rules_cc/blob/main/cc/system_l...

I'd still recommend building everything from scratch (and understanding the relationships and graph of your dependencies), but if your build isn't that complicated and you want to role the dice on UB, this isn't that hard.

As an aside, the most galling part of bazel's cache key calculations has to be that it's up to the individual rules to implement this how they see fit. The rules native to bazel written in java vary wildly compared to starlark-written rules. On thing you (or someone in your org) end up becoming pretty comfortable with while using bazel in anger is RTFC.

> Probably because building targets with tools outside of the workspace is an antipattern, as it violates hermiticity principles.

Nonsense. Nothing forces you to use tools outside of your workspace. CMake just requires you to set CMAKE_<LANG>_COMPILER_LAUNCHER[1] to point to a compiler launchers, which can be anywhere where you see fit, including a random path within your workspace.

People try too hard to come up with excuses for design flaws.

[1] https://cmake.org/cmake/help/latest/prop_tgt/LANG_COMPILER_L...

It's not out of the box if you need third party tools is it?

Reply on news.ycombinator.com