Building a project shouldn’t be complicated. There should be extremely minimal branching and if checks.
My blog post where I argue this in more detail: https://www.forrestthewoods.com/blog/dependencies-belong-in-...
"I need common.lib, it gets linked into my application at compile-time"
"why does every binary include this common.lib code? Let's pull that out to a shared module so we only have one version of it! See? that saved 35739475749Kb of storage! Everything is smaller and sleeker now!"
"which version of common.lib does this application need? Hmm, but that other application needs an older version. I'll have to hack some way of having multiple versions of the common.lib available"
"ahh, this application needs version 5.2.1 but that dependency needs version 4.11.6! Two incompatible versions in the same application! Who designed this thing?"
"right, let's compile whatever versions of common.lib we need into our binary. It'll make life easier and to be honest there are so many versions of common.lib around that we might as well have a separate version for every application"
"Ah, small problem, I have to compile a binary for my application for each variant of each platform because common.lib actually varies per platform... hmm, wonder if I can link to the system version of that lib at run-time..."
<repeat until heat death of the universe>
Somewhat. Although it's agnostic to static vs dynamic linking. The philosophy goes a bit deeper than that.
> See? that saved 35739475749Kb of storage!
Storage capacity hasn't been relevant for over a decade.
Besides, we foolishly replaced "use shared modules to save storage space" with "build multi-gigabyte docker images because it's the only way to reliably launch a simple program without crashing on startup".
Bullshit. It's especially visible with games. I cannot install more than 3-4 big games on my 512GB drive because apparently all game developers believe that "storage capacity isn't relevant".
AAA video games have massive storage requirements primarily due to textures, and sometimes audio. Which is a totally different conversation. It's not a bunch of redundant static libraries causing your pain.
$100 will get you an 8Tb spinning drive or a 1Tb SSD or possibly 2TB. You can also get an external SSD for the same price. Large game sizes is a real and annoying issue. But even 4 year old consoles launched with a 1Tb SSD.
The Linux installs that run your Pi, router, toaster, security camera and toothbrush begs to differ.
> Besides, we foolishly replaced "use shared modules to save storage space" with "build multi-gigabyte docker images because it's the only way to reliably launch a simple program without crashing on startup".
But... you just said it still is :) By two orders of magnitude now.
Insane? It’s what all of BigTech and every game dev does. It’s tried and true.
> vcpkg, conan, and Fetch_Content fix all the issues
Not really, no.
> Have you ever had a build fail because of a network error on some third-party server? Commit your dependencies and that will never happen.
author's "ideal" solution:
> First, the user clones a random GenAI repo. This is near instantaneous as files are not prefetched. The user then invokes the build script. As files are accessed they're downloaded. The very first build may download a few hundred megabytes of data.
Seems like contradiction to me? Do you want your builds to be offline or online?
I have a feeling that the real problem is that (1) author never worked with build systems which supply their own compiler, like buildroot or hermetic bazel and (2) author only downloaded dependencies from third-party servers.
You can solve all the problems in the blog post by (1) changing build system to download compilers and libraries and (2) adopting some sort of "binary storage" (something as simple as private ftp server + small text file checked into repo with path and checksum + few scripts to upload and download). And it would not require any new major code investment, nor new tooling, nor new VCS.
The key idea is that all dependencies should always come from your local binary server, never from the third-party server. This is a bit of PITA as many modern tools default to downloading stuff from third-party servers, and it could be hard to convince them to look at your local server instead. But once you do it, you get reliability + reproducibility + sustainability, without having to reinvent the world.