Show HN: Xmake, a modern C/C++ build utility
github.com
github.com
When did "Show HN" threads became shark tank? Can people at least check and post about the tool itself instead of discussing CMake vs Ninja vs Meson?
From the 2007 announcement of Dropbox:
https://news.ycombinator.com/item?id=8863
""" 1. For a Linux user, you can already build such a system yourself quite trivially by getting an FTP account, mounting it locally with curlftpfs, and then using SVN or CVS on the mounted filesystem. From Windows or Mac, this FTP account could be accessed through built-in software. """
They tried to sell their 'get public URL' wrappers as the replacement for the functionality, but that only covered a tiny aspect (and OneDrive, Google Drive, etc already offered that stuff) - sharing photos - and even that was slower due to them wrapping the files in some sort of viewer instead of giving the raw file data like the public folder did, while using some sort of hash ID to identify the file instead of the actual filename (this was another convenience lost since with the public folder i could guess the URL to share without copy/pasting).
I never found anything as convenient as Dropbox's public folder while the rest of their offering wasn't anything i cared much about and even then by that time both Google and Microsoft's offers were better (and with Microsoft's available right out of the box in Windows).
They are also the only ones I know that have a Linux application.
That being said I don't pay for their premium service because it seems to expensive compared to OneDrive and Google Drive.
People tend to stick to familiar topics ... as shown in the next Meson comment turned to Python2-vs-Python3-compatibility in a whim.
Thanks for trying to improve the ecosystem around native code building tools. I appreciate it a lot.
Large projects such as systemd and gnome[1] have migrated or have been migrating for years
[1] https://wiki.gnome.org/Initiatives/GnomeGoals/MesonPorting
The requirements say Python 3.5 or newer. That was released in 2015. Outside of major versions, Python has generally been forwards compatible (especially if you're testing). After more than a decade of use, the only grumbles with supporting old Python version is not getting to use new features (or working around fixed bugs).
I say this as someone who still uses Python 2 every day.
I'm glad we seem to be past the big inflection point and hope to put Python 2 behind us. But I can see never changing `python` from Python 2 (just not have one when Python 2 isn't installed).
To the point of the comment thread, that would only be an issue if you're maintaining a Python 2 codebase and using modern Arch. If you're creating a Python 3 codebase you're likely calling `python3` and even calling `python` isn't a problem for you.
If that really bothers you then just create a virtual environment and locally deploy whatever version you want to run. That issue ceased to be a concern years ago.
Yaw super lightweight ...
I haven't used virtualenv very much, but I've been doing similar things for longer than virtualenv has been around. You need a `python` binary on PATH and a PYTHONPATH pointing to where you want to get your libraries from. virtualenv packages this up into a directory in a more user friendly fashion.
Most docs I've seen on Python2/3 advise on choosing one or the other, ideally 3. Which implies mixing should either be done very carefully or with a plan to transition to 3.
I get that some accidents of history have caused headaches. The deep adoption by distros have caused problems when people just want to use Python/libs installed with the OS (as well as moving to Python 3), authoring packages hasn't been great historically, etc.
I don't mean to be rude, but the key to getting people's attention is to get your point across quickly. As soon as somebody starts going all you have to do is install a frobozz and murtle a plinth I really just lose interest.
EDIT - doubting myself I went back up the thread here and this topic kicks off with "meson installation has many dependencies, it is not very lightweight" ("lightweight" operationally defined here for this discussion) - to which somebody replies "it doesn't matter, Ninja includes Python" or words to that effect. "it's not the correct version of Python" is the general gist of what comes next to which our GGP here says "all you have to do" is set up a python virtual environment, which is around the point where I start to feel frustrated and the need to go do something interesting.
So to be clear, the discussion is not about python being lightweight, but meson, and trying to wave complexity away with yet another layer of complexity (even if it's something as well known as python virtualenvs) is not reducing complexity but hiding it, IMO.
I agree long wandering threads are difficult to follow.
I have little interest in new C++ build systems, but will jump in and contribute when I see a familiar problem I want to learn more about like managing sizable Python codebases. Python 2/3 isn't relevant at all to meson and I don't see the docs mentioning any need for version or package management. I've only ever heard/seen virtualenv needed in project management, never in distributed software. Is your beef with Python or any software using an interpreted language? Outside of the 2to3 transition and being deeply integrated in many distros, it has the fundamental problems other interpreted languages have and are handled in the same way.
But to be fair, I usually just clone a Virtualbox machine because I'm used to the workflow. :)
I think the pressure to always be on the latest and greatest is what did for the ruby and perl communities, and this is why python, and to a certain extent javascript are still going strong!
If they promise to never break backward compatibility again in the future i might change my stance though, i mostly try to avoid Python than anything than a simple calculator or quick and dirty scripts because i do not want to write any code that may break in 2-3 years without me doing anything wrong.
Guido said there would never be a Python 4, though he has handed over the reigns so who knows.
As a Python dev, I wouldn't mind a Python 4 if it fixed some of the older / crustier corners of the API.
export PIP_REQUIRE_VIRTUALENV=true
It's the only sane way to use Python.ninja isn't installed on macOS and Windows, and both systems require to first install a package manager before installing ninja, or compile ninja from scratch.
Having everything in a single standalone executable is vastly preferable IMHO.
Meson has no glob support.
Meson does not support any form of remote caching.
add_requires("libuv master", "ffmpeg", "zlib 1.20.*")
add_requires("tbox >1.6.1", {optional = true, debug = true})
target("test")
set_kind("shared")
add_files("src/*.c")
add_packages("libuv", "ffmpeg", "tbox", "zlib")That's not how dependencies are discovered in cmake. Dependencies are added with calls to find_package, and if you have to include dependencies that don't install their cmake or even pkconfig module then you add your own Find<dependency>.cmake file to the project to search for it, set targets, and perform sanity checks.
* No standardization or opinionated design, so you can't share your work easily.
* No sane defaults, so your build system is always fragile, difficult to maintain, and done wrong.
* No best practices, so people keep making the same mistakes over and over.
* Misguided attempt to remain compatible with the steaming pile of legacy they've accumulated over the years.
* Bad documentation, so there's no way to learn how to do things better.
* Steep learning curve with limited payoff, so most people don't bother.
Meson does some of these things better. It's still not pretty, but it's nicer to use than CMake.
You forgot to mention that the language is awful (but that usually doesn't get in the way IME).
The answer is that CMake is mad and full of gotchas. Think of it like the PHP of build systems. Here is a classic example:
https://cmake.org/cmake/help/latest/command/if.html#variable...
CMake actual gotchas I dislike enough to avoid it as much as possible include defaulting to cached information, even after I make changes, and preferring "smart", opaque and even hard to track down scripts to explicit user input. It seems optimized for cleverness and conciseness, at the expense of reliability and required user effort.
It's good to have tooling and IDEs for CMake because CMake is complicated and hand-editing the files is very tedious. But if Meson eliminates the tedium of CMake by providing you with different abstractions then you don't actually need the IDEs or tooling.
I have not used Meson, but other build environments (e.g. Make) don't interact as well with IDEs as CMake does (with the exception of premake, which is mostly dead.
No, that's the wrong question and one that's only purpose is to deflect attention from its shortcomings.
All build tools need tooling because if they are adequately integrated into development workflows they are transparent and easy to use. Cmake meets that requirement, and until other alternative build systems do then they will always be far more complicated.
Another poster has explained the IDE support is about IDEs being able to parse CMake files, and I can say that back in the day before CMake IDEs would parse the C/C++ directly using a compiler to output an abstract syntax tree that they would use. So for example Eclipse has this notion of "Build configurations" which allows you to control how this parsing occurs and which files the parser considers valid and what symbols are predefined. Which is IDE support very much like you're looking for from CMake. I worked with it for several years at my last job to provide support for other engineers working with a Make-based build system.
C++ really benefits from it. ctrl+click on a symbol is much more sane than (re)teaching Argument Dependent Lookup rules to all of the engineers in your organization.
You are very fortunate if you import libraries that just work. This is also true for "modern" CMake.
I'm probably biased as I write and maintain Starlark for C++ codebases on a daily basis.
https://github.com/tboox/tbox/blob/master/src/tbox/xmake.lua
That is, I have a bunch of .cpp files which need to be compiled into individual executables in a folder bin/. I also have a folder inc/ which contains some headers (.h) and those headers possibly also have some associated TU (.cpp).
Now g++ can already generate a dependency graph of headers for an executable. It is then (with a standard Makefile and some supporting bash scripts) quite straightforward to mangle that graph into a list of translation units (namely those files whose name matches a depended-on header) which must be compiled and linked into the executable.
That is, I can simply create a new "executable file" .cpp file in bin/, include some headers therein and when I say make, the Makefile automagically figures out which (internal) dependencies it needs to link in when linking the executable.
Now that I have these "relatively straightforward" scripts and the corresponding Makefile, the incentive to move to another (nicer) build system which would require me to rebuild this infrastructure to fit into this other build system's view of the world is quite low – unless there is some way to do this directly?
Xmake as shown here (and also Meson linked in a sister comment) appear to still require manual selection of dependencies.
Actually, it cannot; and this should be well known. It emits in practice less than half of the information that it knows from path lookup, that a build system really needs to know.
And when it comes to creating a library, it's difficult to infer which TUs should be pulled in or left out since you'd need to see at least representative samples of the use of that library to be able to infer that.
I.e. you just have to put all the source files from which you generate object files that will go in the same libray in a set of directories which does not contain files that will not go there.
Similarly, all the source files for the object files required for an executable, except those that are in libraries, should be in a set of directories.
The sets of directories need not be disjoint, just a given set must not contain files that must be excluded for linking a certain target, as that will make the building process more complex.
Given this constraints, it is possible to write a universal GNU makefile usable for any project, which will generate automatically all dependencies.
For any executable or library you want to build, it is enough to write an extremely small makefile, containing 4 lists (of defined symbols, of source files, of directories with header files and of libraries) and the name of the generated file and its type (excutable, shared library, static library).
At the end you need to include a makefile that is good for any project targetting a certain CPU + operating system combination.
The makefiles per CPU/OS must define a few things, e.g. the compiler used and other utilities, option flags for all, locations of the tools and so on, then you include a unique makefile for all architectures and operating systems.
I have started using this method more than twenty years ago and I have never ever needed to write manually any dependency information.
Whenever I see the huge and intricate and impossible to maintain makefiles that are too frequently encountered in many software projects, I wonder how one is willing to waste so much time with a non-essential part of the project.
From my point of view, building easily any large software project is a problem solved a long time ago by gcc & GNU make, but for reasons that I cannot understand most people choose to not do it in the right way.
Of course having to use in 2019 a programming language which does not implement modules by any better method than including header files is something even more difficult to understand, but I still must use C/C++ in my work, as there is no alternative for most embedded computers.
However, one typo can lead to a confusion because an entire word is missing. In the 4 lists that must be written in the makefile, the most important list, as the other lists can be omitted, is the list of directories with source files (not a list of source files).
For simple projects the list will be reduced to a single source directory. Whenever you add, delete or rename source files, there is no need to edit the makefile of the project.
All changes can be taken automatically into account by GNU make, which can be instructed to scan the source directories for source files for all the programming languages that you use.
This will hopefully change with the introduction of C++ modules in C++20 but until them the best option available to C++ programmers is either manually managing third-party libraries or employing dependency management tools such as Conan.
However, just because it is (like most build system questions) outside the scope of the standard does not mean that it isn’t possible to define some project-internal rules about what gets compiled and linked into what and that the build system cannot apply those rules to take work away from users.
The few external dependencies my project has are installed semiautomatically before any compilation starts.
If you are writing an open source C++ library, even if you support some other C++ build system, chances are you will also have CMake support as well.
While I have no doubt, xmake is easier to use than CMake (just having Lua over CMake's abomination of a language is a great improvement), the fact that so many libraries and tools already support CMake is going to make adoption an uphill battle.
As user, I find the experience with autotools to be much nicer as well. For whatever reason, the interface just seems more intuitive. I mean, ./configure --help will tell you basically all you need to know. An underappreciated bonus is that you don't have to install more stuff just to __build__ some program you might not even want. I've run into more than one project that required a specific version of cmake, which as luck would have it, was not the version packaged with my distro. This leaves you either building another cmake first or finding a tool/library that isn't so persnickety.
Given the choice between trying a project that uses CMake or or autools, I'll choose the autotools based project every time.
sorry what ? I remember hours in my younger years searching which Debian package provided autowhateverflavoroftheday.sh so that I could build $random internet project
> sorry what ? I remember hours in my younger years searching which Debian package provided autowhateverflavoroftheday.sh so that I could build $random internet project
The whole point of Autotools is that distributed source packages can be built by themselves, without requiring any part of Autotools to be installed. They build even on obscure systems that don't have any working version of Autotools.
If you have to install autoanything to build a random project that uses Autotools, either you are doing something wrong, or the project is using Autotools wrong, or maybe the Debian package is using Autotools wrong.
That said, I know what you mean. I've had to seek out a number of different versions of Autotools just to get some things to build. But that is because a lot of projects and/or distro packaging blatantly uses Autotools differently than it's was designed to be used. I don't think Autotools should be blamed for this.
(I don't know what build system I'd pick these days - probably just write the Makefiles by hand.)
yes, it absolutely should be. If a tool is misused, it's generally because it's hard to use correctly. In contrast, if I see a repo with a CMakeLists.txt I know that it's going to be a simple cmake && make.
Citation needed.
Tools get misused all the time. If I use a flat bladed screw driver as a pry bar/chisel/whatever, that doesn't mean the flat bladed screw driver is hard to use.
Imho dealing with dependencies and making project build in one shot is hard in cmake.
Maybe I did things wrong though.
That really depends on what you're trying to do. Cmake's happy path for building an executable that depends only on libraries that already support cmake is very straight-forward.
They would disagree with you.
* Borland Makefiles
* MSYS Makefiles
* MinGW Makefiles
* NMake Makefiles
* Unix Makefiles
* Watcom WMake
* Ninja
* Visual Studio projects
* Green Hills MULTI
* Xcode projects
And more: https://cmake.org/cmake/help/latest/manual/cmake-generators....
Although you're conflating project transcoders with makefiles, nevertheless that's the whole point of cmake: generate makefiles that are used by some third-party program to actually build the software.
There are still many differences in the behavior of different IDEs.
"generate native makefiles" (I prefer Ninja myself) - this is what used to build software
no, you don't
remove from your system make (ninja, msbuild, ...) and see how cmake builds your project
Yes exactly when you remove a piece of the build system the build system stops working.
When I use bazel I need python installed or it doesn't work. That doesn't mean my build system is python. It means my build system takes advantage of python. Same for cmake and make.
But when you remove a dependency of a tool you can expect the tool to stop working. That doesn't mean that the tool doesn't do what it says it does.
Kind of like a container for building? I had to look it up myself.
Unless remote dependencies are used, this is optional.
You pin all dependencies and manage flags and such but via the build system.
target("test")
set_kind("binary")
add_files("src/*.c", "src/*.cpp")
add_cflags("-fPIC", "-Dxxx")
add_cxxflags("-fPIC", "-Dxxx")
The command line arguments just give you a quick and easy way to modify cflags.The people behind cmake already learned that lesson with their modern cmake approach, but it seems the xmake people didn't do a proper review of the state of the art.
Or scons? https://scons.org
I wouldn't find having many build system such an issue if they'd just add a makefile (and maybe ./configure) to call that build system, giving devs a consistent interface and not having to lookup up how to do a simple build.
That's the whole point of cmake. Instead of running autotool's ./configure (which in fact is a whole dance involving autoconf, autoreconf, automake, and whatnot) just run cmake . to get yourself a fancy makefile.
Well most instructions I've seen start of with "mkdir build && cd build && cmake ../src", which is a bit more complicated than just "make" with a default build dir. I'm not sure why they're all like this, I would have though supplying a default build directory is something cmake could handle.
My last and only big project with cmake we ended up with a makefile anyway to drive all the things cmake couldn't do, or that we couldn't do with cmake due to inexperience or some combination of the two. So we ended up with make calling cmake calling make, all because it apparently made it easier for IDE users (it didn't but that was definetly our fault).
- Windows - generates a VC++ project
- OS X - generates an Xcode project
- Unix (other) - generates Ninja files for Debug/RelWithDebInfo
(Typically there's also the option to generate Ninja files on OS X if you like - good for automated build and/or if you'd just rather use some other editor (which is not unreasonable).)
Once it finishes, on Windows you load "build/vs2017/whatever.sln" into VC++; on OS X you load "build/xcode/whatever.xcodeproj" into Xcode; on Unix (other) you change to "build/Debug" or "build/Release" and run "ninja". And off you go. After that, it all just kind of runs itself.
The Makefile consists of basically stuff like this:
.PHONY:unix
unix:
rm -Rf build/Debug
mkdir -p build/Debug
cd build/Debug && cmake -G "Ninja" -DCMAKE_BUILD_TYPE=Debug ../..
rm -Rf build/Release
mkdir -p build/Release
cd build/Debug && cmake -G "Ninja" -DCMAKE_BUILD_TYPE=RelWithDebInfo ../..
plus some ifs to ensure the right target(s) are available depending on host OS.(I've found it beneficial to regenerate everything entirely from scratch each time in the Makefile - ensures you're always working from a clean slate, with no cached variables sticking around from old runs. The odd package does have an unusually time-consuming configuration process, but I've always ended up managing to bypass these somehow - it's possible a future revision of my "process" will have to actually address this properly.)
This process does confuse people that don't read the instructions, as they type "make", some stuff happens, and then nothing. But I've found it to work well enough.
cmake -H. -Bbuild && cmake --build .
Will do an out of source build with the default toolchain on windows/mac/linux. Has the added benefit of being parallel, and working with Visual Studio/Ninja/XCode.
As for distributing CMake, the intention is presumably that it's installed on the system of whoever's going to build the code, like make, or gcc, or whatever.
In general though, if you're just talking about small projects, I have found the easiest way to incorporate smaller librares into a build system is to just ignore whatever build system they are using and re-write the build in the larger build system (even if they are the same tool!). This is mostly because the current state of build systems is so terrible.
What you say is true, but it can be interpreted as either positive or negative features. I would say that code that depends on specific versions of a library or specific compiler options is bad code; and propagating bad code instead of fixing it is not a good idea. Cmake makes it very easy and convenient to ship bad code, as you explain. Thus, it is a force of evil! It allows, even encourages, the programmers to be sloppy without short-term visible consequences.
To be fair, cmake was at a time the n+1 build system but nowadays it's pretty much the only option in C/C++ land.
Conan and Meson seem so much better in that regard.
You can see https://xmake.io/#/home?id=cross-compilation and https://xmake.io/#/home?id=mingw
What would it take beyond https and a well-known site to make you comfortable doing this?
Most package manager use an offline signature mechanism done at build time (rpm, dpkg, nix) and do not rely on HTTPS security for anything else than eyes-dropping reasons.
Relying purely on HTTPS is insecure. Nothing guarantee you that your source / script / package did not get hacked / modified between the time you uploaded it and the time your user downloaded it.
This is not hypothetical scenario, it already happened in the past with sites like sourceforge.
Normally the packager and developer are different people so there is a second set of eyes to at least give a cursory glance to the changes. It takes more than a single compromised account to publish malicious changes. There are of course a million exceptions and caveats to this and it's not perfect, but it's better than allowing developers to push code directly.
The vast majority of users would trust Homebrew (for example) to not do something like that.