Build2, a Cargo-like dependency and build manager for C/C++
build2.org
build2.org
More concretely: As a go-to example for these repos I always look for Google's protobuf library. Most projects I work on depend on three or four high-level libraries which, in turn, depend on protobuf; if even protobuf isn't there (as is the case at cppget.org) then it's hopeless that those higher-level libraries will be.
So far, the best native package manager I've come across is vcpkg [1]. I hate that it's made by Microsoft, and I hate that it uses CMake, but there is no easier to build and use e.g. gRPC on Windows. They recently added Linux and Mac support (static linking only). Edit: for fair comparison, it has just over 700 packages.
This is also why I'm not terribly interested in these language-ecosystem package managers. Good C libraries, such as I use in my own C projects, keep a stable backwards-compatible interface for years.
This is factually wrong. I'm able to compile things as complicated as clang/llvm, firefox, and LibreOffice relying mostly on dependencies installed through the distribution's package manager.
Sooner or later you will run into an issue where your development work will depend on a (version) of a library that will break an app you depend on.
Or you will need two versions of a library for your previous and next release which both need maintenance.
Nor do they provide reproducible builds and other features that are needed for serious work.
Distro package managers are not suited for development and dependency management. They might work most of the time, but that is not good enough.
https://github.com/chromium/chromium/tree/master/third_party
the main point of their build system gn (similar to bazel) is to have explicit and distribution independent control of all dependencies (typically tied to specific versions).
LLVM basically has no system dependencies as well, instead it implements a lot of platform abstractions within the repository (https://github.com/llvm-mirror/llvm/tree/master/lib/Support), the remaining dependencies are mostly build dependencies such as python 2.7 and cmake.
Same story for Cryengine (https://github.com/CRYTEK/CRYENGINE/tree/release/Code/Libs) and Unreal Engine (https://github.com/EpicGames/UnrealEngine/tree/release/Engin...).
Linux distributions simply duplicate a lot of work because they are based on a flawed and outdated distribution model, outdated because the main argument for shared libraries disappears with > 1 TB commodity hard drives. The approach taken by the projects above is also the only way to do cross platform development, simply because both Windows, macOS and Consoles don't have the concept of an operating system package manager. (If you are using homebrew for development on mac, you will run into a lot of trouble sooner or later).
Sure, you can run both archive-based and version control-based (git) private repositories without any restrictions.
My comment was more responding to the claim that this wouldn't be a useful tool without a large body of open-source packages, which I disagree with.
I find that the second limiting factor I hit is being able to easily interop with other languages and to cross-compile for other platforms. Buck makes that super easy, too.
Looking over the docs, it seems like a rather all-encompassing build tool. I might have to try it out for the one project that I'm still paying attention to which is C++ based and uses Hunter[0]. I see a lot of references/examples talking about using this for Java projects, but I haven't come across anything related to C/C++ (other than the cxx_\* calls, which appear centered around bringing in a native dependency). Any suggestions for a good quickstart project out in the wild or a larger open-source codebase (C/C++) that uses it?
[0] Or it's less official, but more commonly used name, "f!cking hunter", followed by loud, anger-ish sounding noises.
The problem is that there isn't a nice way in C++ to separate modules into discrete, importable components the same way that you can in Python, JS, and others. This leads to fractured imports and strange build processes.
Maybe Clang or GCC could be modified to output these header only libs. The other option, is to use the package specific compilation tooling but generate WASM. Then use WASM as the module system.
But I agree with the CMake part. CMake is abomination. CMake is diabolical. But it is not vcpkg problem. It is C++ community program. They chose CMake because it was the facto build system for most packages.
I'll poke a little at your assertion that it's "never the build tool itself". Unfortunately, in my experience, it often is. Have a nice fight with Hunter in a project or two. It's got a decent, though still somewhat small, number of packages[0], but that doesn't make it useful. Every time I reload my PC and clone the two repos that rely on it, I invariably spend a day sorting out something or another (last time it was signature related). It doesn't help that the tool is tied to CMake, these are the only two repos I use that rely on CMake and I have actively avoided learning enough about it to not require several Google searches when I need to do somewhat simple things, but that's the rub. Cargo is simple, even npm (by comparison) is simple even for projects with great complexity[1]. Hunter, by comparison, has me asking every single time I have to update a dependency or install fresh from a clone ... what problem is this solving for me? How is this less manual than managing dependencies with a clever zsh script[2]?
As far as vcpkg is concerned, I agree, though I'm a Microsoft fan[3]. It generally works -- and unlike Hunter, which uses CMake as well, I haven't run into the kind of hair pulling, four-letter word spewing fits using it ... at least not to the extent that I did.
To the authors: Really, though, please please keep investing in making better dependency management tooling. If it's good enough, there are enough of us that will make "unofficial packages" because it takes less time to figure that out than it does to figure out how to fix the broken dependency management system we're stuck with.
[0] https://docs.hunter.sh/en/latest/packages/all.html
[1] I'm certainly not saying it's not without its pointy corners, but I'll take those sharp edges any day over some of the C++ build tools I've played with.
[2] Which is the only "build dependency management" system that's worked for me -- I literally "lynx --dump" the releases page, parse for URLs, download and script out each build dependency manually ... I even have a set of functions that manage to do some of this in a relatively well abstracted manner given the circumstances. Granted, these are single platform target apps, but so are the handful I have that use vcpkg.
[3] Well... to be fair, my one Windows 10 install is in a virtual on an openSUSE Tumbleweed installation, so maybe I'm not a fanboy, but with their movement towards open sourcing a lot of their tooling and frameworks, there's not a lot I can complain about, personally.
I just wish Hunter was part of CMake by default b/c CMake should be able to load other CMake projects and resolve dependencies. It itself has all the information internally to make this really easy - ie. targets, interfaces and dependencies. Handling this from outside of CMake is just a much harder problem and you end up having to restate a lot of what's already in the CMakeLists.txt.
ExternalProject_Add() is a giant footgun that should have never been part of CMake - I wrote a little description of why here: https://geokon-gh.github.io/hunterintro.html
There are two outstanding issues:
- if you need to link to proprietary libs then the rebuild-everything-from-scatch strategy is deficient - but then things are going to get messy anyways..
- if libA links to libBv1 and libC links to libBv2, then there is no clean way to link both libA libB and libC in to an application. Usually libraries aren't too picky about which version you link - but there is always a chance things will blow up in your face. This is probably going to be solved with flatpaks runtimes or something like that
I remember one of the most frustrating incidents that I had with Hunter was, indeed, a CMake issue of my own (well kind-of) causing. On a one-off machine I was running a build on, I was getting signature failures every time Hunter went to download code. I checked everything (and, of course, there were signature-related things that looked like they might be a problem but probably weren't). It turned out that the cmake binary on this host was not compiled with SSL support. I guess the folks who wrote error messages for earlier C++ compilers wrote error messages for Hunter/CMake, because at no point did any error offer any hint about the binary lacking SSL and it wasn't until I decided to download the source code to the latest CMake (because the packaging system I was using didn't have the latest). I can't remember the completely specifics (i.e. if hunter/cmake relied on cURL and it was libcurl/curl itself lacking SSL support), but I remember I happened upon the solution by accident while reviewing the compiler flags of the current version of the installed bits to figure out what to use to build the latest version and I realized that the URL to the dependency was https and there was no way that was going to work without the --enable-whatever flag required to support TLS :).
So yes, it's possible I am or have been blaming hunter when I should be blaming cmake, or more accurately, my lack of knowledge in using both tools. At the same time -- and I know this is a really unfair comparison -- I support very few things in Python or node, however, the grief I encounter when I return to that tooling is minimal. It requires a lot less out of me to get things working even in complex applications.
It has a very large collection of packages already available, supports many existing build systems (CMake and autotools work out of the box, but you can hook up anything), and can download binary packages for almost everything.
For the next version we plan to implement a CI/CD bot which will poll your git repository for tagged versions and automatically test and publish them to cppget.org (there will be some sort of a control file that you will be able to use to specify which tags should be published where, etc).
Oh
For more delicate maintenance of stable software (which will often need to depend on specific versions, customize builds, etc), it's still a little hazy. Conan looks ok, but I don't think it clearly beats my current solution of hacking something together from scratch in a couple hundred lines of Python.
I don't think it clearly beats my current solution of hacking something together from scratch in a couple hundred lines of Python.
[0] Though I tend to use bash (well, more often, zsh), rather than Python.
[1] And probably all of them have a Boost package...
I think OP is on the same page
* no concept of versions to declare API compatibility
* no easy way to hook into a central package repository
* source files to build have to be named manually in CMakeLists.txt
* header dependencies can easily be wrongly declared, which is never noticed
* the language is hard to work with
* it's quite slow for large projects (10s of thousands of compilation units)
* cross-language builds are a pain (C++ + JS)
I'm not saying this is all CMake's fault, I really like it for the reasons you outlined. At work, we are transitioning to Bazel to get rid of some of the pain points above, although it has different problems.
Having worked on some larger Go and Rust projects, I find the integration of language and build tools to be fantastic to work with and wish the C++ community would come up with a practical solution (probably involving Modules TS) at some point.
Unlike most other programming languages that encapsulate the
build system, package dependency manager, and project
dependency manager into a single tool (such as Rust's cargo
or Go's go), build2 is a hierarchy of several tools that
you will be using directly and which together with your
version control system (VCS) will constitute the core
of your project management toolset. Next-generation, Cargo-like integrated build toolchain for
C++.Meson can download other Meson projects. From anywhere (no special repo). And you can use that as a fallback for pkg-config (best of both worlds).