CMake Announcement from Boost Steering Committee | Hacker News Reader

CMake Announcement from Boost Steering Committee

lists.boost.org

54 pointstobiasrenger68 comments

CMake is a big improvement, but it's a shame they didn't opt for Buck or Bazel. These two build systems are as much an improvement over the previous generation as autoconf and GNU Make were improvements over random ad-hoc configure scripts. Both take pains to ensure that builds are fully hermetic and fully reproducible, support rich introspection, and run very, very fast.

True, but Cmake has a lot of users already familiar with it and who have it pre-installed. So I applaud this change to use Cmake!

It looks like few people are familiar with CMake, others are just copy-pasting snippets of configuration code from googled results. And noone knows how to do it right. The same is with Autotools.

Even if you look at Kitware own products they full of quite outdated and ugly code. I think it's because amount of code and platform-specific hacks in CMake / CPack / CTest is just too hard to maintain and many things will break hard while there was no changes in them for years.

Also copy-pasting sometimes work better simply because very few developers actually able to check how their own configuration work across multiple platforms and endless number of Linux distributions. It's might look easy to make your own configuration, but once you try to at least cover Windows, Mac and Deb-based Linux it's take weeks to debug everything. Instead you can copy-paste someone else module, change few lines and it's will already cover everything you need.

EDIT: typo: on -> once

But Cmake has support for IDEs like Visual Studio. That is a huge benefit compared to Bazel.

Neither Buck nor Bazel have mature support for Windows. And I very much dislike them for the enormous amount of dependency.

But unless you are confined to Windows, that doesn't really matter, does it? Most code for Windows seems to be written using Microsoft's own tools as far as I know, which makes both the people asking for FOSS tools and the people that make FOSS tools to begin with less likely to target Windows.

The beauty of CMake, for me, is that it can generate both Makefiles and VS2015 files. An example that I've posted here before about was a userland USB driver backed by libusb. I wrote it in idiomatic C++11 (or C++14, don't remember exactly), and amazingly I was able to generate a .so, a .dyld, and a .dll from the same source code, without needing a single #ifdef. Built using gcc on Linux, clang on OSX, and VS2015 on Windows. CMake would output Makefiles on the first two, and a VS solution file on Windows that I could build. I'm still amazed by it a couple years later.

That's a pretty ginormous "but unless...". It would not surprise me if more people are building and using Boost than all other platforms put together. I haven't done any Windows development in 10+ years, but it's not like everyone else stopped when I did.

We're in the context of Boost here and it has great support for Windows. Switching to a build system that doesn't have mature support for Windows is simply a non-starter for the Boost project.

That a lot of FOSS projects (understandably) have poor support for Windows doesn't mean that all of them do. It also doesn't mean there isn't demand for FOSS projects on that platform.

> But unless you are confined to Windows, that doesn't really matter, does it?

Yeah, why should 80% of the OS market share mind to people who develop software for a living?

I completely disagree. Other companies do provide excellent support for CMake right now. Take for example android new cmake toolchain support for compiling for android. Compiling your project for Android is as easy as adding a command line parameter.

Neither Buck and Bazel are nowhere near as Cmake.

As someone who works with C++ and cross compiles code to arm. I really appreciate and enjoyed with recent trends in porting codes to cmake build(prebuild?) system .

Buck and Bazel are definitely far better designed than these scary piles of macros like CMake and Autotools, but far from being ready for the same use cases. These new tools are designed for Google and Facebook's specific use cases: building binaries to run on their own servers, or mobile apps, from sources in monorepos (with dependencies copied to repo![1]).

I think even building packages for Linux distros is hard with these systems and cross compilation is barely documented too (however seems that it is supported, even for Windows). There are no discovery of libraries installed in system and features of system (killer feature of autotools).

[1] https://buckbuild.com/command/fetch.html

Or meson, which gnome and systemd now support, among other big projects.

While bazel and buck look great for building Google/Facebook-sized codebases, meson looks great for what most of our projects are.

Personally, I'd much rather see Meson than either one. But CMake is still a solid, mature choice here.

One obvious issue with Meson is that it's not really available as a (statically-compiled, if you want) binary.

I was an early adopter (years and years ago) of Scons, but the Python dependency turned out to be an absolute disaster, purely in terms of infrastructure (especially that everybody had to have a lot of "things" installed).

While CMake isn't exactly great, a) it is reasonably understandable, and b) the boost build system was essentially a single-project-build system, and c) it's a huge improvement over autoconf, and d) it doesn't need a lot of infrastructure aside from what ships with it (to contrast with 'interpreted' build systems).

My only experience with Bazel was building TensorFlow. I can't say that the builds were fully reproducible or very fast. That TensorFlow is a big project excuses the speed (most of the time was probably spent actually compiling), but I had to rebuild three times until setting the flag for GPU support had the desired effect. So whatever magic was supposed to deliver reproducibility was not working. Add to that issues with some Bazel processes hogging resources in the background and I was wishing for some good old makefiles.

make, configure, autoconf/automake, m4 (I know it's just a macro language), cmake, scons, etc, etc, etc.

Can anyone share a good survey article/paper on the fundamental issues related to build systems, or what is the criteria for a good system (other than popularity)?

Is this a tricky issue because it's tangled with configuration-management and package-management, and OS-filesystem-hierarchy in general? (in which case you can not solve this problem without standardization efforts at multiple levels).

I'm not looking whether to chose CMake or its competitors, but looking for insight into the minds of build-system creators (something deeper than just annoyance with existing tools).

(addition of history discussion would be even better)

Section 10.2 and 10.3 of this PhD thesis are pretty good: https://nixos.org/~eelco/pubs/phd-thesis.pdf

Here's a paper titled "Non-recursive Make Considered Harmful: Build Systems at Scale"

It has a pretty good overview of the what the goals of a build system should be, and section 7 compares several different build systems

http://eprint.ncl.ac.uk/file_store/production/226639/BA1AC95...

Thanks to other responses for helpful documents.

Replying to my own comment to add a 4-5 year old HN discussion about an article from 2003: https://news.ycombinator.com/item?id=7220917

edit: Discussion of Guix, a package management system that interacts with a given build system: https://news.ycombinator.com/item?id=4821488

It blows my mind that CMake is the premier build system for C++. I don't miss that dumpster fire.

Too bad that C++ has invested so heavily in modernizing the language, but it can't invest in a sane, standard build tool. :(

CMake is far more than a build tool.

That doesn't really address their criticism, not that their "dumpster fire" label is very detailed.

That's certainly part of the problem.

Or one of its major advantages. Is there another tool that can generate IDE projects, in a cross platform manner, across a wide variety of compilers, operating systems, etc? I'd love to use a tool that looks nicer, but there is nothing (to my knowledge, I'd love to be proven wrong!) that can do what CMake does with the same breadth and transparency.

> Is there another tool that can generate IDE projects, in a cross platform manner, across a wide variety of compilers, operating systems, etc?

You have the problem backwards. IDEs wouldn't need distinct project files if build systems could reasonably describe the project structure out of the box. There is no need for every tool to know about every other tool. This is a solved problem in most modern programming languages.

> This is a solved problem in most modern programming languages.

No it's not. Not even close.

Just out of curiosity what what's your current IDE?

It absolutely is. My "IDE" is vim. It understands projects in Rust, Purescript, and Go by installing a single plugin. Zero configuration. The same works for VSCode.

> It absolutely is.

No, it really isn't, but... have fun whatever you're doing. :)

> IDEs wouldn't need distinct project files

But they do, and ignoring hard constraints imposed by reality doesn't help increase adoption rates.

> There is no need for every tool to know about every other tool.

If you want your product to be supported by an IDE but you don't have any say on how that IDE is developed, the only sure way you have to get your product supported by that IDE is to get your product to output the IDE project files.

> But they do, and ignoring hard constraints imposed by reality doesn't help increase adoption rates.

But they don't, and saying they're "imposed by reality" is not a compelling argument. They need information about a project; there is no reason that information needs to be generated by a build tool instead of the same files that the rest of the toolchain use. They may also need a file to save a user editor preferences, but there is no reason that needs to be the project metadata file.

> the only sure way you have to get your product supported by that IDE is to get your product to output the IDE project files.

This is patently false. Almost every interesting IDE is pluggable, so a single plugin is written and maintained for each programming language that supports a reasonable project metadata file. CMake is the odd exception which generates files for IDEs because the project metadata is buried in the mess of imperative glue code used to build the project in lieu of a proper build tool.

It's interesting that you think CMake is a "dumpster fire". Consider that CMake isn't the only build tool, but has, without question, risen to the top as the premier build tool for C++.

Why do you think that is? No doubt you'll say it's because the other options are worse, but I'll just say that I like CMake. I miss CMake when I use other languages and I can set up a complicated project with CMake quicker and easier than I can with other tools.

As always, it's what you know.

It's absolutely because the other options for CMake are worse. I don't think my disgust for CMake is entirely predicated on familiarity either--I have much more experience with CMake than I do with Go or Rust's build tools, but I still prefer them. Moreover, as far as I know, none of the new languages are shipping with a fully-imperative, stringly-typed build tool with first class support for dozens of compilers, linkers, package managers, IDEs, version control systems, language libraries, frameworks, precompilers, etc. suggesting they've learned something from hindsight. Instead, the field is narrowing on tools with well-defined responsibilities and interfaces. Instead of knowing about (for example) each compiler, build tools know about a compiler interface that compiler plugins may implement. Instead of writing an imperative script to find your sources, you point your tool to a source directory (if the tool doesn't already assume a convention).

Even if 1% of projects need a fully imperative language, it's absurd to foist that on every other project by default. Besides that, I have a lot of criticism for CMake the language, starting with its implicit import mechanism, ridiculous syntax, everything's-a-string, slow execution speed, etc.

There is work on a sane build toolchain (build system, package manager, etc) for C++: https://build2.org (full disclosure: I am involved with the project).

How is build2 as project planning to maintain packages repository for multiple target operation systems? So far all options for Windows I tested looks failed since they didn't get enough traction for 3rd party contributors to maintain packages and library developers have absolutely no interest.

Is using C++ for everything include web interface is sane solution? I could guess reasons for having build bot and web UI in C++, but not sure if it's good way to get more contributors.

I've seen a few "promising" build toolchains; the sad part is that they need to get significant traction to make real progress. Sincerely, best of luck on build2.

What did they use before? GNU make? (on Windows?)

It seems like CMake is quite popular now, with LLVM having moved to it a few years ago.

b2 / bjam, a custom build tool that was bootstrapped via a script that just builds.

If they have a tool that builds it, why require CMake? Why add an unneeded dependency?

One huge advantage: if your project already using CMake then you'll be able to include exact version of Boost library into your project as git submodule. Then it's will be built hassle-free across all of your toolchains and platforms.

This will of course require some proper configuration on Boost side, but no doubt they'll handle it.

Well, partly because b2/bjam is aboslutely awful and obtuse, etc. etc.

I did some semi-serious experimenting with it at one point and it was just... well, awful. It basically did very few things right.

(BTW, don't take this as criticism of the authors of said system. I just don't "we" had learned enough and infrastructure was incredibly far behind what we have now. So I don't "blame" them in the least. It's just a product of its time.)

I'm not sure what it's like to maintain the build system, but my experience with CMake as a user has been great. It can even generate makefiles for nmake so I can do Windows builds from the comfort of the Windows command line, although I mostly use it under Linux. Under Linux, I've found that CMake is much friendlier when it comes to helping you fix your dependencies than autotools-based build systems.

It probably is nicer than autotools, but my experience with build systems for other languages makes CMake feel like a pile of hacked-together macros. My last company used it for a large codebase and we spent nearly as much time maintaining the build scripts as we did developing features.

Please be specific. What other build systems do you have experience with?

(Because, honestly, I've never heard of a "good one", but there are "less bad ones".)

Cargo and Go's build tool are great.

If all you want to build is Rust and Go code... what happens if you have a project composed of multiple languages?

FWIW, I like "cabal-install", the Haskell equivalent of cargo. It also suffers from the same problems as soon as you start to use multiple languages in a project (except C, that's quite well supported). I stress the word "project". If a "project" to you is just a single language then I'm, first of all, happy for you... Unfortunately that's not the reality for many of us, and THAT is why we need build systems.

EDIT: Point being: A language-specific build system can be great, but as soon as you get a big project with multiple languages it leaves you, the Master of the Build, with a big problem because you suddenly have to incorporate every $OTHER_LANGUAGE into the build... somehow. All of that integration is on you and is specific to your situation.

A build system on the other hand leaves each language implementation with a simple problem: how do I integrate with build system X? One that's settled it works for everyone using build system X, so we're not duplicating that huge amount of effort you would put in in the language-specific-build-tool situation.

(Obviously, this imaginary perfect build tool "interface" is highly non-trivial, but that doesn't mean we shouldn't strive for it.)

There's certainly a place for invoking each project's build tool and stringing together the resultant packages into a tarball, docker image, or nix package. CMake doesn't do this; it gives you a (shitty--for reasons mentioned in other posts) framework for building an ad-hoc build tool for your project, never mind that 99% of projects in any language could be built the same way (which would minimize maintenance work and learning curve).

Yes, but "stringing together" == "non-parallelism" | "unmaintainable because shell scripts". This can kill projects. I'm not even joking.

EDIT: Again, I'm not saying CMake is perfect, but then you haven't offered anything better, so... NYAH? Do you have something better? It's easy to complain about build systems (I do it all the time), but not so easy to come up with something better. Btw, have we dispatched the single-language assumption at this point? (Because, unless "your" language is C or C++, it's not going to happen unless POSIX is miraculously displaced).

There's nothing about invoking build tools that precludes concurrent builds or which requires ad hoc shell scripts (although "unmaintainable scripts" is an ironic defense of CMake, since it's subject to all the same criticisms).

Alternate solutions include Docker, Nix, or even plain makefiles depending on what your artifact needs to be.

Posix is unrelated to invoking multiple build tools and assembling their results.

Well, at the moment your best bet would be "Bazel" I think. It has rules for many languages.

I try to do something similar with my own project https://github.com/jachris/cook – it's not ready though, but you can take a look at it.

As someone with no real experience with make/cmake other than "I've downloaded this thing, these are the instructions for compiling and installing it", what are the advantages of CMake over make?

Cmake uses make (among other backends)

Think of Cmake as a portable, high level layer over make.

Over autoconf: Windows support. Otherwise autoconf is superior in every aspect.

> Otherwise autoconf is superior in every aspect.

Now this is a statement I didn't expect. I currently use make for my simple projects and, as commented a few mintues ago, looking for insight into what makes a good build-system (ignoring popularity).

I used automake/autoconf briefly but it always tripped me up (though I should admit I never learned it properly).

Could you expand on why you think autoconf is superior?

I've used autoconf/automake in some pet projects (a combo that's typically referred to as autotools), and I have to say that the biggest drawback of using that build system is its appallingly bad documentation.

To me, the official GNU docs are unreadable, and the only useful and mildly palatable documentation available on the web comes in the form of blog posts published by people who managed to learn how to get autotools projects to work and took their time to tell the world how they managed to accomplish that feat of technical prowess.

Once that initial part of the learning curve is surpassed, autotools presents itself as a remarkably robust and resilient build system which is highly extensible, which supports virtually all use cases that any user may need to implement.

Build systems such as CMake are started as better alternatives to autotools, and they do succeed in that venture by providing better documentation and presenting a smoother learning curve to accomplish basic tasks. Yet, once a project grows out of the scope of basic use cases.... Everything starts to break, while autoconfs soldier on like a champ.

Try cross-compiling with autoconf without losing your hairs, and we'll discuss about this again ;-)

I have the project which compiles for ios, Windows, and Linux. The best feature is that you do not need to edit source files, but just load projects for the XCode, Visual Studio IDEs, etc. And it works well with c, c++,obj-c, swift, etc.

We use cmake to output a ninja file and build with ninja. Result: awesome build speed as ninja knows how to parallelize across directories and targets while not overloading the system.

This is the first time I've thought about Boost Build since leaving my last job about 18 months ago. The selection by a few of Boost Build for internal projects at the company was easily one of the most divisive, time consuming, and frustrating decisions I've encountered. Even after lots of wiki docs and training, no one understood that thing. Probably <10% of devs could do little more than tweak existing .jam files without all sorts of problems. And probably <1% actually liked the tool, but it got wedged into a process and stayed there.

Benefits to CMake:

IDE Support: KDevelop and CLion both support CMake as a flag ship project file.

Why? A CMakeLists.txt has all you need: the project's debug/release build commands, include's for completions, and library files.

Growing ubiquity: More and more open source projects are build around CMake

CoreCLR, Swift, lua, SFML, opencv, cafe, zlib, libevent, blender, protobuf

Some fast facts about these tools:

- CMake is complicated. Sure, there are macros. But it's not a bad thing. CMake's complexity is proportional to what it takes to do the job: finding header files and libraries across platforms, detecting compiler features, and so on.

- This isn't the kind of a job that can be solved by a simple, beautiful 50 line file.

For instance: Stuff like finding SDL2 header files across platforms is hard because OS leave them in /usr/include, /usr/local/include, then within that, some keep them under SDL or SDL2 within that. Ergo, custom modules need to be built to handle the differences that have organically manifested in distros over the years.

- In order to have software binaries built for as many platforms as possible, a build system needs to be able to test that certain functions exist.

For instance, tmux: https://github.com/tmux/tmux/blob/45ee118/configure.ac#L427. The only way for tmux to know if it should "polyfill" getopt is to run a small snippet of code.

If your system has it, it stores HAVE_GETOPT. If not, it has to get fill in getopt at https://github.com/tmux/tmux/blob/45ee118/compat/getopt.c via https://github.com/tmux/tmux/blob/45ee118/compat.h#L352

- Understand the point of these kind of tools is solve a problem: Getting your source to build across a range of platforms, not just Ubuntu!

- Not to mention, CMake has compatibility for various generator outputs. For instance, it can output Visual Studio projects, XCode projects, and even more popular, Ninja (https://ninja-build.org/)

- CMake gives leaves helpful debug output when so you can get help when you're troubleshooting (cmake --system-information sysinfo.txt)

> This isn't the kind of a job that can be solved by a simple, beautiful 50 line file.

As a big CMake fan, I don't think CMake's success has anything to do with their super ugly custom language. What CMake does could be a lot more ergonomic, especially from a typical user's standpoint.

Typical user doesn't need project generation for 10 different IDEs, OS and packages formats with multiple backends and built-in test support. If you drop half of features then I guess it's could actually become a lot more ergonomic, but in this case there likely tools with better support.

Real pain is that a lot of CMake features are badly documented and most of available information is outdated, but this is usual problem at least for open source C++ projects so it's doesn't cause a lot of frustration and no one try to fix it.

We do, while we're not doing anything crazy, ie. support for 3 major OS (Windows, Linux, macOS), 3 major build-system (VC, GNU make, Xcode), 3 different compiler (VC, gcc, and clang), 7 different product packaging (RPM, DEB, TGZ, Windows, Mac, Alpine, Android), as well all kind of debug / release custom setup.

Exactly.

Another point I'd like to add here: CMake basically takes to you what autoconf/automake can do (+ 5/10 years of improvement on UX/UI), but it:

a) doesn't use absurd amounts of slow shell code trying to detect features that no (non-EOL) OS has been missing for the last 20 years.

b) is actually reasonably readable. (Ok, m4 is sort of readable, but it a bit weird in places.)

c) is actually better documented (IME) than autoconf/automake. What people usually do for autoconf/automake is just copy/paste another project/snippet. .. hence leading to the 20+ years problem. (I'll definitely agree that the docs aren't great, but that's at least a problem that can practically be fixed unlike the copy/paste problem of autoconf/-make.)

I would also mention there number of awesome tools exist for CMake (like Android support mentioned before). One of them I used on our project is MXE cross-compiler toolchain [1]. It's basically let you build CMake project for Windows from Linux by adding one toolchain option and it's have almost all popular libraries available. It's can be setup in like 10 commands at most and have pre-built packages for CI.

[1] http://mxe.cc/

Reply on news.ycombinator.com