My gripe with all CI systems is that an an industry standard we've universally sacrificed performance for hermeticity and re-entrancy, even when it doesn't really gives us a practical advantage. Downloading and re-running containers and vms, endlessly checking out code, installing deps over and over is just a waste of time, even with caching, COW, and other optimizations.
The perceived practical advantage is the incremental confidence that the thing you built won't blow up in production.
> even with caching, COW, and other optimizations
Many CI systems do employ caching. For example, Circle.
I feel like 90% of the computer industry is ignoring the lessons of Bazel and is probably going to wake up in 10 years and go "ooooooh, that's how we should have been doing it".
We are ~8 engineers, and yes, definitely. However there should be good buy-in across the team (as it can be quite invasive), and depending on your choice of languages/tooling the difficulty of adoption may greatly vary.
I was the one introducing Bazel to the company and across the ~80 weeks at the company I spent maybe ~4 weeks on setting up and maintaining Bazel.
I don't know about your current setup and challenges you have with your CI system. However, compared to the generic type of build system I've seen at companies of that size, I would estimate that with 50 engineers having a single build systems/developer tooling engineer focused on setting up and maintaining Bazel should easily have a positive ROI (through increased development velocity and less time wasted on hunting CI bugs alone).
Personally I wouldn't touch it with a 10 foot pole. It's an opinionated task runner, with terrible docs, that if you don't configure correctly, will just hurt your dev process.
The core idea behind Bazel is to make build steps truly hermetic, so you know exactly what inputs they are using. This means you can rely on caching, incremental builds, distributed builds and so on.
I'm sure if you've had any experience of Make or similar systems you've encountered "a clean build fixed it". That root cause of that is because somewhere there's a mistake in your build system where you forgot to declare a dependency on something, and it just happens to work most of the time, but maybe one time the dependency changed and Make didn't know it had to rebuild some stuff so the build breaks.
That's basically why almost everyone's CI system builds everything from scratch every time. Nobody trusts incremental builds.
Bazel goes to great lengths to make it so that you have to declare dependencies, otherwise you simply can't access them. That includes:
* Cleaning environment variables for build steps
* Running build steps in sandboxes
* Storing intermediate artefacts in random directories
* Including tools themselves (e.g. compilers) as part of the dependency tree
Honestly it's not a perfectly hermetic environment, e.g. your build steps can still read the current time, RNGs, etc. so you can still have indeterminacy in your build, but it goes a lot further than anything else.
So ultimately the upside is that you can do things like have CI only build and test things that possibly could have been affected by a change. Fix a typo in a README? It won't have to build or test anything.
My current company spend 300 compute hours and 2-6 wall hours on every CI run, even for fixing doc typos. Bazel can prevent that.
There are downsides though - Bazel was the first system to do this so it has rough edges. And all that sandboxing means there is extra effort to make debugging and IDE integration work. Also because it is super conservative about rebuilding things it can do it sometimes even when it doesn't need to. So I probably wouldn't use it on really small projects, like ones where CI time is under 10-20 minutes anyway.
There are a load of newer build systems that use the same idea: Pants, Buck, Pants 2, Please.build, etc. But obviously they don't have the momentum of Bazel.
Fly has a lot of ideas here, and we've also been able to optimize how things work in terms of downloads and as for boot-up speed, it's less than 1-2s before a runner is connected.
I'm just kinda surprised some Docker-esque thing hasn't stuck. Something that works with Docker, but transforms it to all the advantages of NixOS.
CI pipelines are just so rough and repetitive in plain Docker, which is what we use.
This hasn't been my experience. There have been significant issues with Nix since its inception and very little progress has been made. Here are a few off the top of my head:
* The nix expression language is dynamically typed and there are virtually no imports that would point you in the right direction, so it's incredibly difficult to figure out what kind of data a package requires (you typically have to find the callsite and recurse backwards to figure out what kind of data is provided or follow the data down the callstack [recurse forwards] just to discern the 'type' of data).
* The nix expression language is really hard to learn. It's really unfamiliar to most developers, which is a big deal because everyone in an organization that uses Nix has to interface with the expression language (it's not neatly encapsulated such that some small core team can worry about it). This is an enormous cost with no tangible upside.
* Package defs in nixpkgs are horribly documented.
* Nixpkgs is terribly organized (I think there is finally some energy around reorganizing, but I haven't discerned any meaningful progress yet).
I can fully believe that the community is responsive to improvements in individual packages, but there seems to be very little energy/enthusiasm around big systemic improvements.
> I'm just kinda surprised some Docker-esque thing hasn't stuck. Something that works with Docker, but transforms it to all the advantages of NixOS.
Using something like Nix to build Docker images is conceptually great. Nix is great at building artifacts efficiently and Docker is a great runtime. The problem is that there's no low-friction Nix-like experience to date.
I'm quite diehard in terms of personal Nix/NixOS use, but I hesitate to recommend to colleagues as a solution because the learning curve would likely reduce productivity for quite some time.
That said - I do think that deterministic, declarative package/dependency management is the proper future, especially when it comes to runtime environments.
Not only is it difficult to learn (although that's a huge problem), but it's also difficult to use. For instance, even once you've "learned Nix", inferring data types is an ongoing problem because there is no static type system. These obstacles are prohibitive for most organizations (because of the high-touch nature of build tooling).
> This _is_ a concern for a team that needs to manage it
The problem is that there isn't "one team that needs to manage it"; every team needs to touch the build definitions or else you're bottlenecking your development on one central team of Nix experts which is also an unacceptable tradeoff. If build tools weren't inherently high-touch, then the learning curve would be a much smaller problem.
At best a central team would be useful for managing an artifactory/cache + maybe company-wide nixpkgs, but in general singular teams need to decide for themselves if Nix is helpful + then manage it themselves.