Hacker News Reader

Comment by SwellJoe | Hacker News Reader

I've lately found myself wondering why there isn't a C/C++ tool comparable to npm, cargo, cpanm, etc. There's more C/C++ code in the wild than most languages, but it's not easy to find, not easy to install, not easy to update (though that's a problem in the npm ecosystem, too), and not easy to integrate. A good package manager would go a long way toward modernizing the C ecosystem, I think.

View on HN

I think that's because on most Linux-like systems, the distribution's built in package manager fills this role sufficiently. It's hardly perfect, but for many popular libraries you can just install the -dev package or equivalent, then include the headers from the system path and code away.

I wouldn't mind a more dedicated tool though, especially because relying on distribution-specific packages tends to make cross platform support messy (see: heavy reliance on automake tools to correct for platform differences) and because most distributions only include the most popular, widely used libraries, and don't try to support newer or more niche tools.

The distro package managers like apt or pacman don't really fulfill the same task as development package managers like cargo or npm.

The distro package managers try to supply you with a set of end user applications and then provide a consistent set of libraries to make sure the apps run. This sometimes means shipping an older version of a lib to keep an app supported. It's difficult or impossible to have two versions of the same lib, and sometimes there are hacks like python2-mylibrary vs. python3-mylibrary.

Development package managers on the other hand are intended to build your app and all its dependencies. Many versions of the same library may coexist on the same system and sometimes even in the same project. Development package managers typically build from source and do not have a repository of binary packages. And they allow pinning to specific versions with a lockfile to make builds reproducible. This may also include providing a specific version of the compiler/interpreter.

Nix and Guix may be closer to a development package manager than apt/yum/rpm/pacman, but are yet to be widely adopted.

I don't think your parent was endorsing this state of affairs, but merely offering it as an explanation. I think they are correct - the platform package managers are good enough for the purpose of developers to make it hard for any proper language package manager to gain traction. Even though these approaches may be deeply flawed - by making reproducible builds impossible, among other issues - a majority of users are satisfied with it and do not suffer enough pain to move on. Indeed, I'd much rather `pacman -S boost` and be done in three minutes than wait three hours for a package manager to build it from source. Nix and Guix do get closer to the mark since they allow reproducible builds and can download binaries for me but as long as adoption remains limited I'll be packaging too many libraries myself.

I think (some of) the main reasons are:

- there is no standard cross-platform build system and directory structure for C/C++ projects, a dependency manager either needs to be super-flexible and enable to combine projects with completely different build systems (which is impossible I guess?), or come with its own build system and enforce a directory structure (like Rusts cargo)

- complex C++ projects are not very compatible with each other since they use different subsets of the C++ language and the standard library, this makes it often hard or impossible to integrate external C++ code

This is why header-only, or single header/source pair C or (simple) C++ projects make so much sense, they are often trivial to integrate with your own code.

However, there are some movements into the right direction, cmake seems to establish itself as the standard build system, and there's also https://www.conan.io/

It would be better though if the C++ Standard Committee would get their act together and instead of stuffing more and more useless crap into the C++ standard, use the time and resources to establish a dependency manager standard (along with a proper module system), and let the language users move all the 'useless crap' there.

> complex C++ projects are not very compatible with each other since they use different subsets of the C++ language and the standard library, this makes it often hard or impossible to integrate external C++ code

How does this make sense? If a project uses a certain style, that doesn't make it incompatible with another style by any measure.

If a project disables exceptions and RTTI, other projects which uses those won't compile (at least on GCC and clang). Many low level C++ projects don't use std containers, std::string or iostreams, etc...

Dependencies may be compiled and then downloaded as package. In Java dependencies, theoretically, could be compiled during the project building, but they are provided as compiled anyway. It is faster that way. Here you have different platforms, but I guess it could be done anyway.

A package manager for C++ wouldn't be that difficult if there was a sane, uniform build system available.

But there is something actively developed in that direction: https://build2.org and https://cppget.org

In fact, the package manager is pretty feature-complete, just the build system is still heavily developed.

I agree that there is a huge issue with finding a "good" build system for C and C++. There are thinks like Make but they were designed for entierly different mentalities of software then we have now.

Things like Eclipse have, in my mind, greatly altered the perception of source organization in modern systems. I don't think many people think of source as a file but as rather a directory/collection of files.

I'd love to see a build & package management system for C that would be able to:

   * Easily configureable by the programmer for a large set of end goals.

Whether or not I want to build an executable, use a custom linker, build a specific DLL type of a specific format it needs to be easy. One example I've worked on is getting Make to output just a plain binary for embeded system development or operating system development is a right pain in the ass. I have no clue how to do it.

   * Build into seperate directories

Basically if I want to build a sub project and put it's output into /usr/lib or something then I should easily be able to configure that to happen. Most of the time you want to build into ./build/ or ./bin/ or something like that but don't litter my ./src/ with .obj files.

   * Build out of the directory structure

By default this should look for everything in ./src/ or something to find all of the source files.

   * Associate source file extensions to compilers.

I should be able to associate all of a certian extension to a certian compiler. For example .C->GCC, .S->GAS, .BAS->FBC, etc. A more advanced example may be me needing to compile in a lot of user (being non-programmers in my office) tables. Doing so I could make a .tlb format and a compiler for a .tlb. This .tbl compiler will just internally translate this to a bit of assembly, pass it through GAS, and then give you an object file. This can be linked to the end product and reached from any of the source files. You could also do a .* as a leftover so you can build a library that will pack extra data into the executable that can be unpacked at runtime by your program. This can be done by writing a "compiler" that reads all the data into a flat array and implements a "file system". You could then call a standard API from this compiler the same way that you could read that table. Just a thought.

   * Documentation AND Examples.

I cant tell you how many times I've worked with someone on a project and they've said something along the lines of "do X in make and it'll work better" and it has. There's so many undocumented tricks that it's unreasonable. This is partly because of age, how large the project is, and in my opinion how much cruft the implementation has accumulated.

I've had good experience with scons. It can make shared/static libraries, executables, etc. If you need a custom linker, that is just changing one variable in the environment. Out-of-source builds take a little bit of configuring, with VariantDir, but work well once they are set up.

Scons has a number of "builders", each of which is associated with a file extension. So, for example, I could look for all files in the src directory that end with either ".c" or ".cc", compile the ".c" files as C, compile the ".cc" files as C++, then link them all together with the following command.

    Program("bin/prog", [Glob('src/*.cc'), Glob('src/*.c')])

Custom builders can be added, so if you want to add any additional association (e.g. ".cu" to compile with CUDA), you can.

The documentation (http://scons.org/doc/production/HTML/scons-user/index.html) is pretty good as well

C/C++ has wonderful package managers you just have to use Linux or BSD to use them. Linux and BSD package managers sort of invented the whole idea of network package management in the first place.

BSD ports, and the many Linux rpm and deb package managers have source and developer packages. There was a time when Linux had no package managers. They were invented precisely to solve the problems of dependency resolution and dissemination of sources and binaries to a distributed global network.

The fact that there are so many package managers is a feature not a bug also. Just like the many distributions of Linux and BSD is intentional and strengthens the ecosystem and allows for innovation and experimentation.

Package management is also available on Windows with Msys2 (pacman) and a few others.

cross platform package manager: pkgsrc.org

i have successfully used it for BSD, Linux, Solaris, HP-UX, Red-Hat Linux for the same commercial project.

Amitai Schlair: One Weird Trick To Simplify Package Management.

http://www.nycbug.org/index.cgi?action=view&id=10345

Disclaimer: i am not Amitai Schlair

From pkgsrc website.

> pkgsrc is a framework for building third-party software on NetBSD and other UNIX-like systems, currently containing over 17000 packages.

And Fedora/Yum has 18000+ packages.

Thats a hell of a lot of C/C++ source code thats tested and configured to compile with the system compiler.

I think this is a case of not being able to see the forest for the trees. The entire open source community and operating systems development revolve around a set of extremely robust C/C++ package managers. But they are so integral to development at this point that they have become almost invisible.

Other people mentioned nix package managers (rpm, apt, etc.) - I'll also mention nuget is apparently an option for native code too.

Why isn't there a more generic option? Well: There's a ton of closed source C and C++ libraries out there - stuff with license fees and NDAs, tons of fiddly configuration options that are relevant to you - built with closed source and fiddly toolchains with incompatible ABIs for obscure platforms, sometimes with even more fiddly link ordering considerations... to say nothing of additional requirements for distribution. E.g. some of Microsoft's stuff prohibits "xcopy deployment" and instead requires you to run their installers instead - presumably to make their job easier when they want to roll out a security update.

To try to slay this beast with abstraction is to try to slay a hydra. For every compiler or configuration option you decapitate, two more you hadn't considered before will come into view.

Even when you constrain yourself to the single platform package managers, things are far from perfect. I've had nuget packages that can't handle the latest version of VS. I've seen apt lagging several versions behind the latest boost. They weren't built with the specific fiddly #define s that customize this for my needs. Upstream doesn't want your fiddly project-specific customizations, and legal has better things to do than verify you're not violating any NDAs by contributing back upstream...

The "easy" option is to simply get really good at wrangling absolutely terrible build chains.

> I've lately found myself wondering why there isn't a C/C++ tool comparable to npm, cargo, cpanm, etc.

We wrote a thing like this at Mapbox, try it out: https://github.com/mapbox/mason

> I've lately found myself wondering why there isn't a C/C++ tool comparable to npm, cargo, cpanm, etc.

Someone recommended Conan[0] to me the other day. I'm not familiar with it, so can't say more.

[0] "Conan C/C++ Package Manager" https://www.conan.io/

A small book could probably be written on the subject, but the basic issue is that not enough stuff is standardized for it to be a very manageable problem. The situation is sort of like if npm had to support not just Node, but also all the JS engines and object models in browsers and Flash Player.

There is cget which install cmake packages out of the box, so many libraries don't need to be updated to support it:

https://github.com/pfultz2/cget

There is, it's called your OS' package manager. Languages having package managers is a retarded idea, descentrilizing what has always been good for being centralized - package managers. Want something your distro doesn't provide? Add a source to the sources list, etc.

coz c++ itself is architecture dependent. and even lots of c++ code are architecture dependent. c++ has much longer history than js or java. It runs on bare metal machine for all kind of purpose. So it is much harder to write universal useful libs.

Totally. When you consider linking and binary compatibility, C/C++ does well with the Linux package management world that's grown up around it.

Package management is somewhat less useful modern code that tends to self contain things (Java/Groovy/Scala) or where you want to pin dependencies for stability and create unique environments for them (Python's virtual env, Ruby's RVM, or .. and this is kinda a stretch, but Docker containers).

>coz c++ itself is architecture dependent. and even lots of c++ code are architecture dependent.

For one, there are tons of universal useful libs for C/C++.

Second, not all architectures matter equally. Beyond amd64 and ARM, few give a damn.

Reply on news.ycombinator.com