Does C++ need a universal package manager?
pfultz2.com
pfultz2.com
Moreover it emphasizes reproducible build and seems to handle dependencies in a sane way.
Nix has very sensible defaults and defining a Nix expression to install a cmake or autotools package is a breath. But even projects with just Make or no build system at are easy to install, you just need to copy the output to the /nix/store in the installPhase (to $out). And I can easily define Python or any other language dependencies in the same consistent and concise syntax. I have no fear anymore I won't be able to install dependencies in the future. Any other dev just does `nix-shell` in the project root and they are sure they are ready to go. I even used to hate the Nix syntax (as opposed to Guix) but at some point it "clicked" and now it makes a lot of sense.
Only drawback is no Windows support. Maybe someone at Microsoft could help with that? It would be so sad to have the community converging to a suboptimal solution just cuz Windows... :/
Does Nix make it easy, or even possible, to bring in dependencies from legacy sources on the network? For example, my company has an FTP server full of carefully-packaged, versioned libraries in tarballs, and it would be great to be able to use those, rather than having to repackage everything.
The documentation is good but it can sometimes be a bit rough. There are some good blogposts on how to achieve specific things that you may find when searching around. Also the IRC community seems is quite active, responsive and friendly :-)
Which is false. DEBs and RPMs are not tied to the OS in any particular way. Especially visible in OpenPKG project (it used RPM), which was running on Solaris just fine.
It's more likely that it's OS distribution that is tied to the package manager, not the reverse.
I would like to see how it would work on the native environment for IBM z/OS, for example.
The problem is extra work is needed to support things like cross compilation.
Instead, you want to pass a toolchain file that describes the build environment(including for cross-compilation) to the build system, and it will compile it correctly for the target. This is the way cmake works, but we should have standard toolchain file that can be used across all build systems instead of requiring everyone to use cmake.
For what it's worth, a lot of the glaring issues will soon be alleviated, I'm not that enthusiastic about C++14, C++17, etc. features anyway. C++11 for me is really about catching up with C99.
I don't see any reason for a company to not pay for software licenses.
They don't offer upgrades for free.
Even on macOS eventually getting a new version implies an hardware upgrade to get hold of the newest version, thus indirectly paying for the compiler.
The problem is that the unique combination of ubiquity, expensive upgrades, and relative lack of substantial innovation (which would offer an additional incentive for upgrades) means that MSVC shops can be simultaneously willing to buy licenses one year, and unwilling to buy upgrades another.
I don't like your inquisitive tone and persistent uncharitable reading of my comments, and I will not engage any further without an outright apology.
Good day.
The VS Community license is fine for small companies, and I don't think the licensing fee is a major issue for larger companies. 'Small companies' being defined as those with less than $1 million annual revenue and < 6 employees using the software concurrently.
In my ideal world, nobody would bother with using the MSVC compilers (or they would finally make upgrades free for the compiler at least), and we'd all be running the latest stable or LTS versions, and have similar standards for compiler features and performance. Unfortunately, there is a market incentive for some companies to use MSVC, and there is a market incentive for them to not upgrade that MSVC license until they absolutely must.
For those on the cheap side, Microsoft had the express versions for quite some years now.
"*Visual Studio Express 2017 is available for Windows Desktop developers. This will be the final version of Visual Studio Express, and there will not be a UWP or Web offering of Visual Studio Express 2017. We encourage all users to look at the free development options offered by Microsoft such as Visual Studio Code and Visual Studio Community for future development needs."
Community has license restrictions that were not in place for express versions.
There is also a build tools package now which has the compiler without an IDE, but i'm not clear on the licensing for it.
Any larger company can surely afford professional licenses.
When I started on IT we had to buy everything, just like on other professions.
The problem with funding open source projects is that monetization can kill the community. You either have a dual license scheme or a support contract scheme where a few guys make money and the rest doesn't.
The problem is that no matter how far MSVC goes, those with old licenses for one reason or another tend to bring down the lowest common denominator. I would have no qualms buying a VS Professional/Enterprise license and upgrading it if I were in that position, but I can not very well stand on my pulpit demanding that everyone else do so.
The great thing about the free compilers is that there are enough disparate incentives in the world to fund their development. AMD, Intel, Samsung, Qualcomm, ARM Holdings, and basically everyone else who designs and sells CPUs or GPUs, or uses either compiler extensively enough, contributes to LLVM and/or GCC for the purely selfish reason of making sure the code that is generated makes their platform or software product look good.
The efficiency is very high, because nobody has to sell or market the compiler itself, because the compiler is a tool to ensure the marketability of their hardware products. When AMD wants to sell GPUs to the deep learning community, they base their work on LLVM because it is cheaper for them. They put it in the open because it makes it easier to consider using ROCm in your deep learning or image processing workload; because on the long tail of rare bugs, it is more likely that the customer will fix the compiler than the vendor, even if the vendor is being paid.
I don't like the tone or tenor of the responses to me in this thread at all, you folks are making it out like a) I personally don't pay anyone for compilers, or b) I don't think it costs money to develop software. In reality, I just recognize that the recurring licensing cost of MSVC encourages people to use old compilers with lower feature sets, which in turn encourages open source projects to accommodate those lower feature sets way past their due. With the free compilers, yes the majority of people are freeloaders, but you can at least be expected to have the most recent version (and obviously there was enough money in it to make that happen!).
MSVC is not a better compiler, by basically any metric except Windows integration, than either GCC or LLVM. LLVM and GCC are built entirely on the selfish interests of the union of its users. They are free to freeloaders, valuable to exploit, and always fresh. What exactly is not to like here?
If you spend all your time belittling and making light of people, you'll never hear what they have to say for what it is.
We have less than 5 devs but failed one of the criteria listed here: https://www.visualstudio.com/license-terms/mlt553321/
> Other platform vendors also sell their compilers.
Also, no, Apple's platform compilers are free (though obviously somebody pays for them), and so are the platform compilers for the major free operating systems GNU/Linux, FreeBSD, OpenBSD, Solaris. Sure, you can buy ICC or XL C/C++, but it isn't the only supported way to build and link against those platforms.
Sun Forte surely wasn't free.
While you can install gcc on other systems, usually doesn't support all features available on the OS vendor's compiler toolchain.
Specially on commercial proprietary UNIXes, mainframes and embedded systems.
Unfortunately, it is still terribly easy to write non-portable code with C++. I think this is the main factor why we still don't have a widely used package manager for C++ but your mileage may vary.
That is mainly for binary distribution. Most C++ package managers currently are source-based because binary compatibility is not a easy problem(there are some package managers trying to solve it though).
> Unfortunately, it is still terribly easy to write non-portable code with C++
Yes, it easy to call into a platform-specific API, so cross-platform portability is limited by the testing resources for the package maintainer or library author.
For all of these reasons it is not possible in the general case (the one that package management needs to target) to simply distribute binaries.
Linux distributions are not a working example here because the libraries are built for the use of the other packages in the same repository. Needs of third parties are never considered because finding library configurations that work for the packages in the distro is hard enough.
A platform independent package manager for C++ must effectively degrade into a build system that rebuilds all dependencies from source. And it must allow build parameters for each step to be defined by the dependent libraries or programs, so that the result is something that can be linked together successfully.
Regardless, the point is that all of this is technical and has nothing to do with Mozilla's love of open source.
That's how Cargo works, and it works great.
Haskell has most of the same problems and Stack handles them easily. Each source file in a project can use different language extensions, all transitive dependencies are compiled from source, and there's even CI to re run tests whenever a package is bumped.
Which will (given enough code being built) lead to incompatible flags amongst dependent packages. Compilers may or may not, at build or link time, give sensible error messages. It would be a very sophisticated package manager that could warn you of these sorts of conflicts at specification or installation time.
Your development, on the other hand, will depend on a different set of dependency libraries (and this must be reproducible!). This may include stuff like having several different versions of a library (for different projects or different branches or build configs of the same project). This set of libraries will sometimes conflict with what you need for the apps the OS provides.
Because you never want to break your day-to-day applications supported by the OS package manager, you'll need a different way of installing your depepdency libs. This is what language package managers do.
Right now for C and C++ the alternatives are 1) bundling it with your source 2) use version control (e.g. git submodules) 3) make install --prefix=$HOME/foo and configure build for that 4) single header file libraries. None of these work particularly well compared to a proper development package manager like Cargo.
IMO you use submodules only for libraries not already included in your package manager.
When developing software, you need both installed. Not every distro has -dev packages to install headers.
> IMO you use submodules only for libraries not already included in your package manager.
This approach is full of problems, most importantly that this isn't reliably reproducible (without using docker images or something for a reproducible environment). Your package manager may have different libs or different versions of libs than another developer's package manager.
But the bigger issue is version requirements mismatches. Say your web browser (or other must-have app) requires libwidget-1.0, but you're writing an app that needs libwidget-2.0 or libwidget-0.8 or both (e.g. your latest supported release was 0.8 and next release is 2.0). You can't install 2.0 or 0.8 because it would break an app you need for daily work.
Using your distro package manager for development is a partial solution at best.
Really? I would be surprised if every distro didn't have a mechanism to install the headers and static libraries for all of its dynamic library packages. How would the distro's own build process work otherwise?
> But the bigger issue is version requirements mismatches.
This is the killer. Distros are based on the idea that there should be only one version of any package installed at once. A development package manager needs to be able to handle many.
This feels like it should be relatively straightforward to add to distro package managers, though. You would need a package-space concept analogous to a container - a consistent set of packages separate from the host's, and some way to run a build process within it. So i can say:
dnf create-pkgcontainer ~/dev/myproject
dnf --pkgcontainer ~/dev/myproject install zlib-devel-1.2.8
dnf pkgcontainer-run ~/dev/myproject make
It's a pain in the ass when you need to enter a "blessed" environment to build your project. By "blessed", I mean a container environment, a chroot or just some environment variables that need to be set.
> dnf pkgcontainer-run ~/dev/myproject make
The issue with this kind of builds is that they integrate poorly with your $EDITOR (or IDE). When I run a build, I want the results to be piped back to my editor so I can comfortably jump to the files with errors and warnings in them.
And it's not only the build, it's also running your application, running the debugger, static analysis, linters, etc. Requiring a special environment will introduce friction to all of these.
Additionally, your suggestion of using "dnf" would require every developer in the project to use an OS where "dnf" runs. Builds should be reproducible as much as possible without depending on the OS.
Development package managers solve lots of these problems, although having a package manager for every language is an issue.
apt and yum are completely hostile to reproducible builds.
> in a way apt and yum are package manager for c++
To accelerate the build we support caching and sharing artefacts
Would love to hear your opinion on that
I believe buckaroo uses integrated builds so its likely you would need to supplement it with other information such as source files, but hopefully the core information could be reused.
The biggest time sink when porting a package was to research a projects requirements and dependencies. To our surprise only a handful projects do this correctly.
If we could agree on a standard that captures the dependencies and requirements of a C++ project correctly it would be a huge win for the community.
Currently there is no buildsystem that enables users to describe the requirements of an project correctly.
We choose to support buck as a buildsystem mainly because it allows us to describe nicely the structure of a project. However like any other buildsystem it does not capture the full essence of a C++ project.
For instance there is no way to describe the set of compatible compiler and linker flags.
Many build systems evolved to be turing complete so we can write logic to handle various combination of options(eg. no-rtti) but this makes it really hard to reason about projects.
It turned out to be quite challenging to determine if two projects are compatible with each other - in fact it turned out to be as challenging as solving the halting problem. Is this really necessary?
I analyzed now over 300 c++ projects and I still haven't seen one project that truly needed a touring complete buildsystem to be build (Not even LLVM&Clang).
So yes - I'm convinced a metadata file as you suggested is the way to go.
We want to steer away from standardizing a build system for now. The uses cases are enormous to try to tackle. Instead, it would be simpler to standardized the description of the build environment so the package manager can pass that to the build system.
Hopefully, build systems will be updated to read the package metadata so it can properly consume the dependencies correctly.
> So yes - I'm convinced a metadata file as you suggested is the way to go.
Good to know we are on the right track.
A recent failure: I use some projects which fail to compile with clang or more recent gccs but happily compile with gcc4. Homebrew/linuxbrew don't let me have multiple gccs. I use a server which supports the module system and covers gccs from 4 through 7, so I don't have problems there, but I can't compile some things at all on my laptop.
Similarly, CMake can work well, but it often breaks for me when I try to build some other peoples' projects, and is much harder to reverse engineer to fix than Makefile.
Some of my pain is from the embedded side where most libraries I use need a special set of flags and are compiled to static libraries or directly into the binary.
Nobody needs another tool like CMake that just goes "whatever bro GL, here is the compiler/linker command line variable" once you need to do something beyond specifying input files.
(This is a big area where lots of improvement is possible. I want to kill someone whenever a linker just spazzes out with "undefined reference". YOU ARE THE MACHINE, GO THROUGH THE LIBRARY SEARCH PATH AND TELL ME WHICH ONE HAS THE SYMBOL.)