Clang 5 in a Docker container for C++17
solarianprogrammer.com
solarianprogrammer.com
The description states: "Running Clang in a container has the advantage that it is light on resources and won’t mess with your underlying OS.". Containers aren't really light on resources if the host kernel is not Linux and not messing with OS is easily attainable by installing the binaries in a custom prefix directory (e.g. ~/bin). This is especially true for compilers which have minimal or zero dependencies.
I can see containers are sometimes useful and/or a convenience, but you're right it is currently some weird cargo-cult. I don't find it that lightweight either; it can waste serious amounts of space storing all the different images and intermediate states in Dockerfiles. At least on FreeBSD I can use jails with ZFS snapshots (and dataset compression) to keep space wastage to a minimum, and have it quickly freed when I'm done with them without any extra effort.
The fact that one can install a package in some OS means that somewhere, one or more people sat down and wrote a recipe for building and packaging it, and then hopefully performed some minimal QA on the final product to make sure it actually works.
This is not trivial, particularly not if multiple versions of a compiler may need to coexist (because of other packages depending on a particular version), and it needs to be done for each combination of compiler version and OS.
To me, examples like: "just run <insert package installation command here>", expecting a package to just magically install itself on some OS are the symptoms of a cargo-cult.
Rather, I think that using containers is more like outfitting a cargo container with a fully stocked and staffed workshop, then shipping the container and personnel to where the client needs it to be. The client then only needs to supply the storage space for the container and the goods that need processing (a shared directory in this particular case) and wait for the finished result. Sure, some of the workers may not speak your language, they may use different tools, but until (if?) the workshop gets rebuilt and staffed on "native soil", the inconvenience should be acceptable.
Not to mention if I had a dime for every case of people running full-blown distros under docker I would have been a millionaire by now
Granted, the state of C++ on MacOS is embarrassingly bad.
If you're trying to write portable software, that's the easiest way to validate compilation for Linux without: a) using an external CI, or b) running a full VM in parallel.
But probably the real option (c) that you are missing here is the fabulously named "Windows Subsystem for Linux".
Re WSL: If the app you're working on works on WSL, great. It's not an option for many projects though.
Cross-targeting from linux is probably slower ATM (clang-cl is not fast :P)
That said, if you have a single compiler you can use across platforms, and it generates the best code (or within a small enough percent), and enables a suite of cross-platform dynamic error checking capabilities (ASAN, etc), that provides significant productivity/etc advantages for people. Especially when you can easily improve them according to your own engprod priorities.
MSVC is the worst standard non-conforming C++ compiler in the current major compilers. I consider it MS C++-ish dialect compiler.
Only from C++17 on gcc and clang do have an advantage. http://en.cppreference.com/w/cpp/compiler_support
The label worst std support would go to: EDG eccp, Intel C++, IBM XLC++, Sun/Oracle C++, Embarcadero C++ Builder, Cray, HP aCC [Collapse], Digital Mars C++
MSVC has very bad C support, this is their problem, not C++.
See eg here for MSFT explaining about the lack of even C99 conformance, as recently as November 2017: https://visualstudio.uservoice.com/forums/121579-visual-stud...
Meanwhile clang and gcc have had full C99 for a long time and have complete or near-complete support of C11, including optional parts.
C11 is an extension of C99, so you can't have C11 support without C99.
But this nonwithstanding, MSVC doesn't appear have wide support for post-C99 C11 features either. For example, _Generic and t.
Are there any parts of C11 that MSVC supports aside from the Annex K functions, which is optional in C11? Annex K happens to be an extension proposed by Microsoft, so it's small wonder they implement it and it's quite disingenious to talk up microsoft for C11 support in this case. Other vendors didn't think it was very good, so didn't implement that optional feature.
And contrary to your claim, gcc and clang seem to have quite good C11 support.
Really, it's garbage and it shouldn't be considered as C++ compiler.
I write many simple scripts with node just because it works everywhere with minimal quirks. I also have a few incredibly simple web apps which I use to visualize or explore small datasets. It's great to able to open up an html file in the browser and have it work everywhere.
Docker containers aren't an ideal solution, but they make it easier to get things running everywhere. Consider why stuff like Electron is so popular, despite its many quirks.
In what sense? Apple keeps Clang up to date on macOS, and it is essentially no work to get the associated Clang tooling and other C++ tools (valgrind, etc) installed through something like homebrew.
Actually, not especially. The system compiler that ships with Xcode is still quite old and the libraries are quite out of date. For example, though the most recent Xcode finally added -std=c++17 it still lacks core library features like std::variant. I wish they could keep close to mainline but for whatever reason they can't.
quite old, quite old... -std=c++17 was only added in clang 5 released in september, 3 months ago. AFAIK current Xcode is based on Clang 4.
As far as compiler support goes, Clang supports all of C++17 except for P0522R0 which is a defect report (ambiguity in specification) so cannot be implemented as written. Ditto GCC.
However std::variant is a library entity, not a compiler entity, and indeed, the libraries are a bit behind. Neither GCC's nor LLVM's runtime library support the parallelism extensions and have other gaps in areas exotic (memory, unfortunately which I would like to use) and prosaic (filesystem, which though I'd like to use it as well don't really care about). But they have been solid on C++17 core functionality for quite a long time.
C++ isn't as important to Apple as Swift (or even, still, ObjC) and C++17 is pretty new, so one can hardly blame Apple for this, especially since it's so trivial to simply use another compiler.
It's due to issues like this that I do all our clang testing on FreeBSD, where it's a more vanilla build of the llvm codebase, and I treat AppleClang as a wholly separate compiler variant which requires independent testing.
Probably a VM or really good scripts could take care of it, but since we use docker elsewhere people don't have to learn something new that they'd treat as a black box anyway
IIRC even though windows 10 might support docker through WSL sometime in the future, for now it's just spinning up a vm like on mac. Is it possible to have multiple lxss with different base images in windows 10(because then you'd just use a ppa or something)?
So we have a VM which pulls in a bunch of layers. At that point, you might as well just distribute a full vm through vagrant, then your vagrant up is just a one liner.
I dual boot my laptop with macos and linux, but am I seriously the only one with 256GB space that has to worry about how many vms/docker layers get pulled?
I kinda liked gentoo prefix, far superior to homebrew imho. I used to update some llvm patch for various versions[1], but the lack of binary packages and smooth developer experience is what killed it IMHO.
[1]: https://github.com/fishman/timebomb-gentoo-osx-overlay/tree/...
> Docker for Mac does not use VirtualBox, but rather HyperKit, a lightweight macOS virtualization solution built on top of Hypervisor.framework in macOS 10.10 Yosemite and higher.
One of my targets is SteamOS (i.e.: Linux with a particular set of pre-installed shared libraries), and the preferred build environment to target that is inside a chroot. So my SteamOS builds get built inside a chroot inside a docker container. And that was surprisingly tricky to get working! There are some gotchas to overcome around how chroots handle symbolic links inside a docker container. But I've got it all working now. :)
The big benefit is that host-OS-upgrades-for-security-purposes no longer upgrade my build's toolchain as a side-effect, potentially breaking or changing the performance of my build artifacts; it's fine to keep running ancient software inside that docker container that's just going to be wiped out in 10 minutes and never gets directly connected to the Internet anyway. And the docker image can be copied around to wherever I need it, if I need extra build workers or whatever. Once you've got the thing set up once, it's fantastic for scaling up your build server if you eventually want to do that.
I'm sure with certain languages/runtimes you can leverage the package/dependency manager to serve the same purpose, but I imagine having a docker image for this would mean everyone's using the same base OS with the same installed library versions. This way anyone who wants to contribute to your OSS project can clone the repo, run a simple command to initialize the dev environment, and start hacking right away.
As an aside, I completely agree with your sentiment about trying to learn about the actual tech rather than just being sold on some sales pitch targeted towards less tech-savvy crowds. The best "solution" I've found for this problem myself is to educate myself as much as I can on the fundamentals of the problem space and try to reason about that decision myself.
For example, I haven't done much of a deep dive into docker yet but have spent a lot of time trying to learn as much as I can about the fundamentals/primitives provided by operating systems and kernels. I feel this way I can always reference back to basics and make a better decision of whether something is gimick-y or not.
Lately I've been so frustrated with people selling these "solutions" yet not having any understanding of the how things work that I've spent the last 3 years trying to learn as much as I can about infrastructure primitives (TCP/IP + general networking theory, database theory + database engines, and OS/kernel internals). It takes a lot of time but I definitely think it's worth it since it's a lot harder to be "fooled" when you actually understand things from the bottom up, for some definition of "bottom". It also helps that I work in this space professionally, so the investment in learning these things literally pays off.
Anyway, that's the end of my rant : )
For building, you can create an image that is ready to build your application, then run it as needed. After each run, unless you've set up some persistence, it goes back to the pristine state. If you create lots of almost-identical environments, you can use layering to save space (but space is cheap, so...)
If you want to ship an application that way, you just take that a step further. Create an image ready to run the application. Just remember that storage is, by default, ephemeral. It's a config option away, but annoying if you forget it.
I like that I can just ship an image to someone and it'll work on their machine. Great for our OSX developers.
The interesting functionality is the use of namespaces, which are ways to create isolation of {IPC,networking,users,filesystems,processes} between the containers. Docker didn't invent them, they've been in the Linux kernel since forever, but Docker has made great use of them.
Volume mapping is useful too. Most of what it does is achievable via bind mounts, though, volume mapping is just cleaner, IMO.
That said, I've found the most usage out of Docker from the networking side of things, as extraordinarily convoluted as it can be. I have a legacy application that cannot run on any port other than the one that's hardcoded in the application. It tries to connect to itself immediately on startup, too. Obviously it prevents itself from running another instance... by killing the old instance when the new one starts up. Putting it in a Docker container let me run 12 of them, side-by-side, with the ports mapped externally to different ports, invisible to the application.
Just an FYI, I'm not one of the converted. I think the current fad of "Dockerize all the things" is a bit silly, especially when I start seeing people doing things like putting ssh access for the host machine in a container. I have no interest in their tools other than Docker itself. I'm not a fan of their whole "one process per container" suggestion, and basically ignore it whenever I want (you should make sure you have something running as the init system in your container if you do this). But it has pretty much replaced chroots for me, no small task.
Finally, if you want some good examples of what it can do, look at the images from https://hub.docker.com/u/linuxserver/
For example, Ubuntu 17.10 provides clang-5.0-3 and g++ 7.2.0. Installing clang 5 there is as easy as
sudo apt install clang-5.0
For die hard everyday C++ development, one probably does not want the C++ compiler run in a somewhat restricted container and experience all the potential trouble and limitations.
However, containerizing the compiler is a good way for providing a standardized build environment, because one can pin the exact versions of the compiler, libraries etc.
I have also seen people moving all their not-so-important stuff and/or the components that they do own maintain themselves out into Docker containers, in order to keep their base systems lean and clean. And so they can get rid of the containerized things easily should they become irrelevant. Plus keeping the containerized things relatively stable and unaffected by changes to the base system.
Hell, even on CentOS 7 you can install devtoolset-7 and get GCC 7.2.1
We build our code on MacOS with a simple Makefile that builds all of xcodebuild, g++7 (from Homebrew) and llvm-5 (again via homebrew) without any fancy standing on our heads. Works out of the box and doesn't interfere with the system tools.
If you can't control what's on your PATH variable and your compiler options a container won't save your build process.
Solarian programmer seems to be a talented C++ coder.
Thanks for the projects.
The Docker post was a niche thing. There's other posts for getting a system running with Clang 5 on other platforms: https://solarianprogrammer.com/2017/12/13/linux-wsl-install-.... Another cool post about C++17 and raspberry pi a few days earlier: https://solarianprogrammer.com/2017/12/08/raspberry-pi-raspb...
Sure, I wouldn't use Clang in a Docker container (heh). But there are some points. If building from LLVM+Clang from source, installing files/headers and such can mess up systems royally. One docker container could theoretically test drive clang everywhere without mucking up a system.
$ pacman -Ss clang
extra/clang 5.0.0-1 [installed]
C language family frontend for LLVM
Why do you need a container for that? Can’t you just `brew install clang`, and when you no longer need it, uninstall cleanly?
Today, serious people mock those who refuse to distribute a whole operating system along each program.
A year or two ago, it used to spin up Virtual Box.