I cannot write software outside of Google anymore
twitter.com
twitter.com
Not trying to start a flame war here but… who cares?
I’ve published open source while working at faang myself and they all have internal build tools, version sets, etc.
Plenty of employees still do tons of open-source.
He's still making a valid point that dependency management & build systems for C & derivatives are a shit show.
autotools is great, meson and bazel similary. but people quirking around with cmake alone, or conan or other toys cannot be helped much.
I assume you are joking. Autotools is a slow-as-molasses system (running autoreconf could take half of your build's wall time!) designed for 1980s unix wars (with support for anything not resembling 1980s unix C programs being an awkward bolt-on), complete with an unusually hostile syntax and multiple levels of inefficient interpreters, all optimized to encourage cargo-cult copy-paste programming. It was barely tolerable at the turn of the century, when everyone used it for lack of a good alternative. For a while, its one saving grace was decent support for cross-compiling. But using it for a new project today is insane.
In fact, I just built my own artifact server where I store vcpkg exports and remote Bazel dependencies. That way, statically linking in opencv is as easy as http_archive with a template BUILD file.
Mediapipe and TensorFlow use it, for example.
Did you write Borg or something?
Google made and open-sourced Kubernetes themselves. Yes it's based on Borg, but it was an internal project first.
[0] https://fortune.com/2021/10/29/faang-mamaa-jim-cramer-tech-f...
If they’d have kept referring to Google instead of Alphabet, it could be MAGMA.
I should tell Jim Kramer. Maybe he’ll make that change official.
Everything from the build system to the coding style oozes "NIH", even seemingly simple things are overengineered to fit the "Google way", and god forbid they use any standard tools or even ideas from outside Google.
It wouldn't be a problem if the Google tools would be great, but they feel clunky and as if they've been frozen in time 10..15 years ago.
It’s like one of the biggest problems that seems to be a plague related to build systems is that there is a point somewhere around the level of a makefile that a very large number of programmers can understand but is hard to maintain so various companies build infrastructure on top of the makefile that is semi custom.. then another engineer comes along and builds abstraction on top of that. So you end up with this house of cards that depends upon python 3.8, etc etc then finally there is a makefile.
I do wish all of these whacky build systems would be required to have some choke point where it prints to stdout “generated makefile at path = /……, continuing in 3 seconds… running make -f xxxxxx”
You choose whatever build system you like, create a single-line "install.sh" which runs some "apt install" command, and you are done -- things just work.
The main downsides are that packages are somewhat old, and that your users need the have the same distribution. But containers help a lot with the latter problem.
Edit: I suppose e.g. Debian is generally for binaries, but what I'm pretty sure source packages still work the same way? There's also Gentoo, and maybe Nix.
I hold to a very similar sentiment. I sold Bazel on my blog like this:
> Our builds will be pure functions from source to deployable. This will represent a durable evolution the software engineering field, and Bazel will become as essential as git and source control. Like git, we will often find it annoying. We will punch out comments about how it’s API is not quite right, or that it’s too hard to learn, and we will likely be right. But we will never suggest going back.[1]
Unfortunately Google is not throwing its weight behind Bazel enough to overcome Bazel's significant up-front costs. Maybe in time...
1. https://thundergolfer.com/build-systems/software-engineering...
Nope. It’s unfortunate that Bazel is written in Java. There is no overcoming that. You have to have been isolated at Google for years to be blind to that.
When AT&T dissolved the group that had been maintaining a bunch of unix software, and other people had to try to pick up maintenance of ksh, they discovered a little thing called AST.
> You can use any 3rd party open source library you like, as long as you use their build, package management scheme, compiler, what have you. But you cannot then use any other open source library.
Most everything I've seen will happily accept whatever I've installed into /usr/local/lib and /usr/local/include. Yes, you have to build the thing first. But it's gotten so much easier & better, generally. Sure there's configure, cmake, make, different toolchains... but most of the build tools are pretty flexible & configurable in pretty standard ways. Way more projects are toolchain neutral. (That said, webassembly/emsdk adds a whole new depth of complexity to everything.)
And meson is kicking ass & taking names everywhere, is so much less of a "write it yourself" bag-of-tools and so much more "it just works" framework.
The bigger takeaway here is that working at a large company, for any length of time, tends to sink you in (rather than lock you in) to their ways of working and it can be rather difficult to step into the wide open source world afterwards. It is not out of laziness, but out of learned ways of working and domain knowledge.
It is difficult to extricate yourself from "The <Company> way"...sometimes with panic and dismay at the sheer amount of options out there, that simply aren't as interoperable as "The <Company> way" was. This is a legitimate and valid pain point that professional software engineers, working in companies, face in their careers.
A smaller but equally important thing to note is mention of the number of people working on the build systems internally, that make and maintain interoperability between both internal and 3rd party libraries and tools, to create a smooth build system.
Someone has almost certainly packaged the thing you want with Python bindings.
I know some people manage to do it, in the same way that some people manage to write complex software that uses many libraries in C++. Are they just smarter than me? Or have they simply {discovered, bought, created} a superior toolchain?
To be fair though, the reason PyTorch exists is because Yann Le Cun used Torch, and presumably FB thought a Python version was better than teaching engineers Lua.