Problem: There are N confusing build systems with arcane rules and you just want to compile your program.
Solution: Design a simple, straightforward build tool.
Problem: There are N+1 confusing build systems with arcane rules.
Solution: Design a simple, straightforward build tool.
Problem: There are N+1 confusing build systems with arcane rules.
I’m happy to see people inventing new build systems, programming languages, game engines, ORMs, or anything else where the conventional wisdom is to use the existing tools. It may be crazy optimistic but we need people to keep these skill sets alive.
However, solutions pushed by corporations with grandiose claims do strike a nerve with me. If only because they are typically pushed with the idea that they will spread through some technical superiority. I'm much more open to this getting more use strictly from who is pushing it, not what it is capable of.
* Asana * Databricks * Dropbox * Google * Huawei * Lyft * Pinterest * Stripe * TwoSigma * Uber ...
This is not a new project that needs pushing.
In large, most of the clones you are talking about are because this is a google pushed product. Chromium specifically is because google pushed it in.
Regarding everyone at google wanting this, before entering industry, most folks never used a proper build system at all. It is not uncommon for college or earlier users to just use whatever their IDE does for them. Such that, yes, any automation is welcome after that.
None of this is to say it isn't worthwhile. Heck, they have enough manpower on it that it should be a good solution. It still hits a sour note with me, though.
Gmake, cmake, scons, waf, whatever - non of them provide hermetic deterministic builds, track compiler versions properly, etc.
The only build system I had used which could, with crazy effort, provide some of those features, is iirc ClearCase (or whatever it used for version control and management), and it was painful in every way and still didn’t enforce it - just had enough to allow you to do it with great effort (which gmake/cmake don’t even do)
- Why would you clone an open-source product because Google is pushing it?
- Why are some of the clones older than Bazel? Bazel was open-sourced on Sep 8, 2015, going by first tag, and Buck two days later, so Buck must have been in development for a while. Pants was late 2014, and Please was early 2016.
> Regarding everyone at google wanting this, before entering industry, most folks never used a proper build system at all.
Could you give an example of a “proper build system”? Even if this applies to most folks, Google hires so many people, and there are enough old-timers there (hell, Stuart Feldman was a VP of Engineering) with industry experience.
One of the fundamental problems here is that for large enough projects or multi-project repos, it becomes burdensome just to load and evaluate all of the build rules. The inevitable result is that you have to split the build rules across multiple files for different parts of the tree, and people have been doing this for decades, this is nothing new. If you look at existing build systems, there are very precious few that will actually traverse the dependency graph well across these different subinstances. Recursive make sure doesn't do this very well (see Recursive Make Considered Harmful, 1997). IDEs usually do a good job of this but they only solve a narrow set of problems to begin with.
Once you start with the goal of solving this particular problem well, my claim is that your solution will have some surprising similarities to Bazel (unless you ditch some other objective like incremental builds).
People left google and wanted to copy blaze, hence pants and buck were created.
Then google released bazel, an open source version of blaze.
My hunch is it is a little of both. Combined with a giant dose of not caring about existing solutions. Which is not to be scoffed at too heavily. Being able to say "I don't care" about existing users is a huge asset that any startup should not discard. One of your biggest assets is the lack of external assets to be liabilities.
Still leaves me sour to see the hubris that there is something intrinsic to this. There likely isn't.
Technical debt is when you prioritize short-term gains and pay for it with additional maintenance burdens long-term. In my experience, Bazel adoption can be the opposite—replace your custom scripts with Bazel, make the investment now, and deal with less maintenance later.
I’ve dealt with some really painful in-house build systems, and migrated some of them to Bazel. That was paying off technical debt.
1. A user named gnus-migrate [1] says it well: "I think you completely misunderstood the concept of technical debt. It was just a metaphor created to explain to non-technical managers the need for refactoring. Saying all code is technical debt makes the term meaningless. / I think that you're saying that even clean code needs to be maintained. While that is true it has nothing to do with the idea of technical debt."
2. Eleenrood responds: "Actually it is logical consequence of Cunningham['s] idea. If you assume that overcomplicated solutions generate debt, you can also assume that best solution generate also some amount of debt. Just way smaller than non-optimal solutions. It actually simplify it, cause you don't have to come up with magical line between "debt clear code" and "code adding technical debt". Instead you weight code, which adds more technical debt than other. Your best solution in this case is the one with smallest technical debt weight."
I will admit my preference, at the start of writing this comment, was for the first. It just sees nicer to define perfect code as 0 technical debt, right?
On the other hand, I agree that all code has maintenance costs. (At the bare minimum, from time to time, someone has to read it to revel its perfection and convince themselves that it still remains untarnished given changing business requirements!) So, more points for definition #2.
Next, let's compare software development to factories and machines (what could go wrong with this metaphor?). Yes, running and maintaining factory equipment has ongoing costs. But I would not say that planning for maintenance costs is the same as incurring debt. Hmmm, more points for definition #1.
On the other hand, if a factory is given the choice between (a) buying outright and doing its own maintenance versus (b) renting equipment that includes both the cost of the equipment and the maintenance, you would expect in a perfect (idealized) market that both would cost the same. ... (Slight tangent: Accountants are sticklers about how companies account for costs. Some systems differ in how owned equipment and rented equipment are treated. I'm not expert, but I have a healthy, painful respect for these considerations.) ... All of which would suggest that sooner or later, one way or another, you have to pay for that maintenance. Call it a cost or call it debt, does it really matter? In both cases, if you want to make widgets, you have to pay it. So, points for definition #2 for acknowledging that code requires maintenance which, like it or not, implies a debt/cost.
So, I can see why people prefer either. I kind of like to embrace the tension, so I like both.
But if you need resolution, it can be found! :) The difference between these views is only a matter of the baseline.
Another Arbitrary Table
technical debt technical debt
option definition 1 definition two
------ -------------- --------------
perfect code 0 300
imperfect code 400 700
Of course, this works for addition, but not so well for division. Since division by zero is undefined, I tend to prefer definition #2!
[1]: https://www.reddit.com/r/programming/comments/8w8s03/all_cod...
The core is exactly the same.
After all, there are only tools. If you need a better tool, build it and ignore the naysayers. Everything is an experiment. If it inspires people, the better.
Yes, a big design requirement is to be polyglot, but if that's all you want, Make has been around thirty years.
Bazel attempts to solve the problem of performance and correctness in large codebases.
Most build tools do not scale well.
Did you ever use it, or at least see what a bazel build rule looks like [1] before calling it arcane?
[1] https://docs.bazel.build/versions/1.0.0/tutorial/cpp.html#un...
E.g. How easy is it to integrate a new C++ compiler into bazel? (and I don’t just mean what you get by changing CC to point somewhere else. I want the full integration with the bazel build system in order to get those hermetic builds that are the whole point of using bazel in the first place).
How easy is it to build things which are not in the set of blessed languages? What if I have a Haskell component? Can I easily integrate that into my bazel build? Last time I tried the answer was definitely not.
If you can use the pre-defined BUILD rules then bazel is pretty great. If you can’t and have to start wading into the weeds of bazel’s hinterland then it rapidly stops being so. If you’re inside Google, then you’re probably fine because a blaze/bazel engineer is ultimately only an email away. For the rest of us outside the Googleplex, extending bazel is an extended exercise in frustration in my experience.
You can also define your own rules in a subset of Python (by wrapping other rules including genrule()), so adding new languages should be simple enough.
It’s just a SMOP, right?
what a standard, used by a whopping 2% of the C++ community : https://www.jetbrains.com/lp/devecosystem-2019/cpp/
- I couldn't find an easy way to list targets
- in particular, a way to show me the path to the build artifacts (so that I could inspect it in my editor)
- I couldn't find a way to do build configurations (especially intersections of build configurations)
- it didn't seem great at handling non-file build steps (set up a cluster / server / whatever so I can run my tests)
- the documentation was slow to load and hard to find things in (you basically had to use search, instead of browsing a good table of contents)
As I said, though, I'm a newcomer; it's quite possible there are good ways to do those things, I just hadn't managed to find it in the documenting yet. That and the issues are by no means unique to bazel.
I also do like some aspects of it (like enforced out-of-tree builds), but that's off-topic ;)
Non-file steps: normally in bazel you would consider the server setup a part of the test. So `blaze build :my_test` creates an artifact, and blaze run or blaze test runs the artifact which does non-hermetic stuff like setting up a server.
bazel query //some/package:*
bazel query //some/package/...