Comment by manquer | Hacker News Reader

Comment by manquer | Hacker News Reader

Parent Full thread

Can't comment on Bazel specifically, but having worked with both nx and turbo, the bottleneck was usually network and disk IOPS rarely compute.

Even fully cached outputs needs to fetched and read from a remote server[1]. A step n-1 outout fetched from remote cache server need to written to disk and then again read by step n[3] - all disk I/O and network bound operations.

10s may be achievable/realistic goal in the Java/C++ world where Bazel normally seen. In TS eco-system most people would be over the moon to get into ballpark of 1-2m for a decently large monorepo.

We should define Build more clearly here, if you mean running just transpile/compile steps or the full series of steps that includes tests (as the linear post here is talking about). It is hard to see even a small sub-set of a large suite of test that require a virtual DOM or a real browser can run in 10s or less.

[1] Typical for say managed CI setup .

[3] Common run-of-the-mill frontend + backend stacks in different languages etc.

If you don't need to rebuild anything, bazel can fetch only the final artifact (not the intermediates) from the remote cache.

Also, if you have persistent CI workers with a persistent bazel instance, you save on some network roundtrips, but that's obviously harder to set up and make bulletproof.

Typically you need the intermediates to compute if you need the next one , so you cannot skip to final until you have the intermediaries.

The final asset/artifact is rarely small either. even best optimized artifacts can be few hundred MB docker image or more commonly multiple image layers running GBs in size .

each step is a network pull then recompute cache if stale and keep going till end .

We solve it 2 ways in the Bazel ecosystem: for the intermediate artifacts, we only fetch the digest (hash + size) of the blobs to calculate the merkle tree forward. The blob itself can stay on the remote cache server.

For the bigger final artifacts, we support using Content Defined Chunking (rolling gear hashing) to only fetch the missing chunks between incremental builds. Binaries executable with stable layout benefits from this quite a lot.

We are definitely not done with all of the improvements here. But since all the major AI labs are using Bazel, we know that the tools can support “Agent Scale”. https://webazel.dev/

> calculate the merkle tree forward. he blob itself can stay on the remote cache server.

Not sure how that would work with building say a docker image, reproducible builds are pretty hard problem to solve, and caching intermediate layers is not always simple or even doable, we typically still need to publish to a registry which is not the cache server.

The Bazel ecosystem builds container images not by using Dockerfile, which contains non-reproducible primitives such as RUN and others. We do it by actually constructing the file trees and tarballs manually, then using them to compose the JSON manifest and indices. This is done via smaller hermetic and reproducible Bazel actions and thus enables the ecosystem to scale way beyond what alternative BuildKit-based solutions can.

https://www.youtube.com/watch?v=biYXmAv4Ppk&t=314s should be a good talk to study up on the matter. The speaker is now working at Apple.

Bazel can even skip the final artefact!

well for test runs you kinda need the binary to run, but you're correct, if the job is just "does this build" no download necessary.

Reply on news.ycombinator.com