Hacker News Reader

Comment by boris | Hacker News Reader

I don't believe we can get to thousands of packages by packaging a substantial part ourselves -- we will just get bogged down in this instead of building a decent toolchain. Instead, the "secret" I believe is to make creating, testing, and publishing C/C++ packages as easy and frictionless as possible. This is how you get to thousands of packages and this is what we are trying to do.

View on HN

This is exactly what linux distributions do. Big ones with lots of contributors package around 10k libraries themselves. Small one-man shows package at least a few hundred common c / c++ libraries.

This is also why I'm not terribly interested in these language-ecosystem package managers. Good C libraries, such as I use in my own C projects, keep a stable backwards-compatible interface for years.

Any sufficiently large software project can't rely on the distribution to provide anything for them. The macOS and windows application distribution models are much saner in that regard. The goal should be a self contained, easily reproducible build with minimal external system dependencies. This is why build systems like bazel are so great: They force you to be very explicit about what you depend on and give you a stable and reliable way to reproduce your build once you've specified that.

> Any sufficiently large software project can't rely on the distribution to provide anything for them.

This is factually wrong. I'm able to compile things as complicated as clang/llvm, firefox, and LibreOffice relying mostly on dependencies installed through the distribution's package manager.

It is not, just take a look at the third party directory of chromium, where it tracks all its third party dependencies

https://github.com/chromium/chromium/tree/master/third_party

the main point of their build system gn (similar to bazel) is to have explicit and distribution independent control of all dependencies (typically tied to specific versions).

LLVM basically has no system dependencies as well, instead it implements a lot of platform abstractions within the repository (https://github.com/llvm-mirror/llvm/tree/master/lib/Support), the remaining dependencies are mostly build dependencies such as python 2.7 and cmake.

Same story for Cryengine (https://github.com/CRYTEK/CRYENGINE/tree/release/Code/Libs) and Unreal Engine (https://github.com/EpicGames/UnrealEngine/tree/release/Engin...).

Linux distributions simply duplicate a lot of work because they are based on a flawed and outdated distribution model, outdated because the main argument for shared libraries disappears with > 1 TB commodity hard drives. The approach taken by the projects above is also the only way to do cross platform development, simply because both Windows, macOS and Consoles don't have the concept of an operating system package manager. (If you are using homebrew for development on mac, you will run into a lot of trouble sooner or later).

Clang, Firefox and LibreOffice are all likely to be in distro packages themselves, so their build dependencies have to be there as well. Building something of similar complexity that isn't already packaged would be a fairer test.

The distro package manager is there to provide your day to day apps and the libraries to support it.

Sooner or later you will run into an issue where your development work will depend on a (version) of a library that will break an app you depend on.

Or you will need two versions of a library for your previous and next release which both need maintenance.

Nor do they provide reproducible builds and other features that are needed for serious work.

Distro package managers are not suited for development and dependency management. They might work most of the time, but that is not good enough.

The distro package manager is there to offer a menu of compatible known-good dependencies that you can readily adopt. It's a ton of work you don't need to do. If instead you bundle random stuff committed last month, you should have a compelling reason for beginning to maintain a half-baked distro all by yourself.

Which is all fine and dandy if one only cares about Linux distributions, the world is bit bigger than that.

I leave this feedback so that hopefully it may be useful to you, but when I was investigating rewriting my build infra a while ago, I looked at build2 and didnt use it for this very reason. I know that I'm one person, but the fact that the gp is the #1 comment tells me that other people had similar thoughts as well. Something that might help kickstart this is if a few of the big packages were up there: openssl, libprotobuf, opencv, google test, etc. If those really big ones were there, that would draw people to add in the other packages that go with those thereby expanding the available packages. But now, since theres nothing, you would have to do everything by yourself in a new system which is daunting for anyone. (Also, by maintaining those packages, you learn what the pains of maintaining the packages are and so as you "get bogged down" you can solve those problems making it easier for others to maintain their own packages)

Yes, all valid/good points, thanks. We are packaging external libraries that we need ourselves (libmysqlclinet was "fun" -- it is actually a C-library with C++ implementation details). And getting some "critical mass" of packages done by us is also something we are considering.

Is it at least able to consume CMake projects so we can mix and match until more packages are available?

Yes, see https://build2.org/build2-toolchain/doc/build2-toolchain-int...

Reply on news.ycombinator.com