Single binary executable packages
notes.volution.ro
notes.volution.ro
The new Self-Contained Deployments feature in .NET made our lives infinitely easier as a B2B software vendor. We no longer have to have sweaty-hands conversations with the customer about which .net frameworks are installed or even allowed (?!) in their environments. I literally click my mouse... 4 times in visual studio and have a zip file containing everything we need to send to the customer.
We also moved to SQLite recently, so that one zip file is literally the entire kingdom. You could grab a blank VM from Azure and our product would startup and run on it without any extra bullshit.
There are certainly tradeoffs involved and it's not always the right decision, but these days I usually try to publish software as a single file until I can't.
Agreed. We are going to move to proper single file (or as close as we can get) as soon as any sort of meaningful AOT compilation is available:
https://github.com/dotnet/runtimelab/tree/feature/NativeAOT
https://github.com/dotnet/runtimelab/issues/248
There is a "PublishSingleFile" option, but that is just a zip file in disguise.
I don't quite agree. These days (as of .NET 5), if you publish a single-file app all .NET libraries live in the executable and do not need to be unpacked at runtime. Native libraries still need to get unpacked at runtime (if your application uses any; many don't) but I haven't found that to be a big deal in practice.
Thanks for the correction/clarification. I had not realized this was a thing now. Will give this option another look.
But in general, you put it in an archive.
(Yes, that's inconvenient; but when you're coming from an OS where you already had to do that — e.g. macOS 9, where you had to use StuffIt Expander to distribute executables in order to ensure they retain their resource forks — switching from StuffIt to zip was a wash.)
These OSes (NeXTStep, macOS) usually integrate some support for mounting disk images, though, and so executables packaged for distribution are usually packaged by putting them in a disk-image. To further decrease the friction (i.e. avoid having to say "copy this thing out of this directory into your /Applications dir", they usually make it so that if you run the app from inside the image, it copies itself to your /Applications dir, and then relaunches from there. Sometimes it even unmounts the image when it does that.
Given that most people get programs from websites, though, there are also special techniques built into web browsers to make downloading executables on these OSes more transparent and less requiring the abstraction of a disk-image. E.g. Apple's https://en.wikipedia.org/wiki/.XIP format, where any .XIP downloaded in Safari will be automatically+transparently integrity-verified and unpacked, with the result being that it seems like you just "downloaded a folder" somehow.
And your end user desktop application just doesn't need to be built that way!
Experimenting with Uno framework, I was able to build a Skia Linux GUI that built to only 2 files, the program and a skia library.
https://www.joelonsoftware.com/2004/01/28/please-sir-may-i-h...
I'm not sure it really matters, in the end. How often would people want to run PowerPC-era Mac software on modern hardware natively if they could? The HN audience will include a disproportionate number of people who do, so you guys aren't included ;-)
I haven't used a Mac in several years but it seems like the whole Apple mindset is more appliance like, that if you bought old software you just keep running it on the original old hardware, and given the problems with Microsoft's eternal backwards compatibility approach (applications and the OS bundling forever ancient code and keeping ancient interfaces around forever, and ending up with a disproportionate number of API calls numbered foo2, foo3, foo4ex), I'm not sure either way is strictly superior to the other. There are trade-offs with both.
I think some creative people, e.g. writers, might like to be able to continue to run their favorite old word processors if they could. Every so often you'll hear about a writer who's still using Wordstar or WordPerfect or some other old software because they know it inside out and it does what they need.
Of course, using old hardware has plenty of disadvantages (the increasing scarcity over time as parts become impossible to acquire) but even now you can find power macs for cheap. Your average ebay listing is pretty ridiculous but I snatched a G4 ibook there 2 years back for $14 USD, essentially in like new condition.
Korg released an audio DSP playground in a PCI card called the OasysPCI which never got OSX drivers, so I have a MDD PPC Mac to run that. There are probably better things running natively today, hey instruments are things that shouldn’t be obsoleted, since they all have their own sound. Most of the rest of the ancient software I run is run under Wine (which worked quite well up to Mojave, but became difficult when Apple killed 32 bit support) because Microsoft has done better at retaining backwards compatibility. So now I have a laptop permanently stuck on Mojave.
And it's not an academic one - a large percentage of somewhat older Mac-native Steam games don't even start.
It includes much more recent software too. It seems like half the mac games on Steam are 32-bit binaries published before October 2019 that no longer run on modern versions of macOS.
I dread the day we have to put together an installer for it. If we ever decide to use GTK on all platforms we'll have to since the GTK developers don't seem to think static linking is a useful thing for them to support.
The GTK folks oppose static compilation, and that's a perfectly fine point of view because they can support their opinions quite well. However, without an official "Microsoft visual c++ runtime" equivalent, distribution of GTK apps is just so annoying on Windows! You can get the entire GTK library suite just fine on most Linux distros but when it comes to non-Linux platforms, you're left to your own devices, and that means every platform comes up with its own solution, incompatible with the rest.
The fact is that is usually done in the Linux ecosystem: if you have access to the source code, what is the pro? It's just simple to recompile the software targeting the release of the operating system you are using. By dynamically linking system libraries you will get smaller executable and save RAM since they are shared (not only on disk, but also in memory, that is what shared means!)
I dread every time I need to install a (older) package from source. I think around a quarter of the time it turns into a multiple hour adventure of frustration, only to discover the library it was missing is called $NAME-dev on my distro...
You lost me at this part: if the dependency is easy to install and the issue is you just don't know that all of the headers are in -dev packages this seems like the easiest problem to never have again. Really: right up until there I was rooting you on.
For code, you can compile by specifying the correct C standard. Unless the build system is badly written and doesn't add the correct -std=cXY to gcc of course (but it's trivial to fix).
Regarding libraries, that is a real problem. Because to compile and old software you should use old version of the libraries, and they are usually not easy to get (and then it's a pain to compile them, and put the correct environment variables to make the software you are building link to that libraries and not the one in the system).
I'm always able to compile a software, even an old one, on my system, and it never took me more than a couple of minutes. But I recognize that I'm a pretty experienced Linux user, so sure for the average user having a binary package is more easy.
The waste of time to compile anything is staggering. And more often than not I give up on failure after 2h. Because one of the recursive depency is impossible to build. Or I got tired of git cloning or wget'ing 50 differents stuff.
Sometimes it will ends with: XXX is missing, required version mismatch, install correct version, your computer crash.
If you look at something like `apt-cache show podman | grep ^Built-Using:` (Output: https://paste.debian.net/plain/1225449) on Debian 11, you will see why. Imagine a few of those components shared between tens of packages, and security problems discovered im some. It's got to be any package maintainer's worst nightmare. "Traditional" distros run by volunteers and in participation of small-ish companies will not be able to cope with that kind of need for rebuild churn - and yet they will live amongst us for many years, if not decades, to come.
Second, shouldn't the majority of the time rebuilding packages be dedicated to testing each application with the new library? Would you just skip that step if you used dynamic libraries?
In my experience, non-developer users don't care about package managers (they want it to be invisible) but do care about one click install. The only way I've found to satisfy those users is to distribute applications outside the distros' package managers and statically link everything as much as possible.
It saves me and my users time and money. Package management with inverted dependency trees is user and developer hostile, and it's why shit like podman and docker have to exist in the first place.
You most likely don't have experience creating and maintaining packages for every distro that exists, expecting you to be able to do this would be silly. Typically the users or developers of the countless different distros will be who handles packaging your software though, not yourself!
As a developer you just need to provide the source code, use a standardized build system such as make, cmake, meson, cargo, etc, and a list of libraries that your software depends on. If you do these things, creating a distro specific package for your software will be totally trivial for whoever feels like doing it and contributing to their distro's repo. Most distros have decades worth of tooling accumulated to make packaging software that uses standard build systems absolutely trivial!
Optionally, you can provide a flatpak image (or similar) for users who want to circumvent the system package manager.
And in order to reach that "thousands of users" my users do need to be able to actually use my application (open source or not); this means either I or them has to do all that work...
So yes, one needs to "have experience in creating and maintaining packages for every distribution that exists"; or as I've proposed, just skip all this hassle and provide single binary executables for the platforms (not distributions) that one supports.
I am not against providing a static binary or image (or whatever) by the way, this is probably the best thing that you can do!
I was just trying to clear up the misunderstanding about distro specific packages and who creates and maintains them since most people don't have much experience with the process.
It's also worth nothing that this isn't really a one way or the other deal, you can provide a portable method and people can package your software eventually. I think the two styles work very well with each other.
Even something that seems innocuous like, "oh sorry it's not available for that by default, here's a tarball" is too much friction for user applications.
Flatpak is the closest to a solution we have, but it has its own issues. It's infinitely superior to "just use the standard build tool like $(N different build tools) and list your dependencies, then pretend it's someone else's problem to solve!"
----
I'm not endorsing GitHub, I'm quite neutral about it, however at the moment it does offer a "standardized" experience. So much so, that whenever I see a project announced on HN and the link takes me to a landing page, I immediately look for a GitHub link and switch to that.
In fact, the article brilliant describes one of them:
> Modern software distribution: AKA "actually free with no-ads apps store" of the Linux and BSD worlds.
That's it. That's the advantage summed up in a single description that wasn't even intended as a description of the advantages of the current way of doing things!
The article has a list of "cons", but it utterly fails to consider the possibility that this overwhelming advantage of what it calls "modern software distribution" is simply and inescapably tied to the maintainer-oriented approach that currently underlies it!
Attempts to create modes of distribution that are even slightly different from maintainer-distribution (think Google Play) suck ass. For all its faults, F-droid is a vastly better platform because it insists that the software must be built and distributed by an F-droid maintainer. Stepping even further away into the realm of developer built opaque binaries is begging for chaos and misery.
As you say, it's ultimately a security concern [1]. The article claims that a change is necessary because of "increased usage of languages and run-times that don't fit the current build model", "some of these new projects have large numbers ... of dependencies", but these are themselves problems with modern software development. Even putting aside distribution, bloated nodejs dependency trees create security vulnerabilities. The inability to develop software without pinning exact versions of your dependencies (which then need to be manually upgraded by the developer) creates security vulnerabilities and fragility. These are problems, not good reasons for changing our current way of distributing software!
I'm convinced that there are some (like me) willing to die on this particular hill. Come what may, even if half of the developers out there switch to Go and only ship static binaries, we're going to continue working on and using traditional Linux distributions with maintainer controlled software. (To be clear: Go and static builds are warranted in many cases. For example, closed-source rarely updated software like games, and software that is "deployed" rather than installed. But these are not the base case for Linux distributions.)
Further reading: http://kmkeen.com/maintainers-matter/
[1] https://blogs.gentoo.org/mgorny/2021/02/19/the-modern-packag...
I'm not sure Google Play is a great comparison here. Package managers like Homebrew and Scoop are probably better ones; when I just want the latest version of a CLI tool, they make that experience much better on macOS and Windows than on Linux (I know Homebrew sorta supports Linux, but it's still early days).
I see the success of these programs as essentially stemming from the insertion of user interests in the form of a maintainer-like process. Sure, they're still working with the binaries, but the actual process of installing and managing these binaries is controlled by users, for users: https://github.com/ScoopInstaller/Main/tree/master/bucket
This means that you get moderation and in many cases modification to the behavior of the program. In a freeware environment like Windows that's full of shitware, at the very least you can in many cases strip out the ads. That's absolutely not nothing, but at the end of the day it comes from a group of user-maintainers stepping up and saying to developers that no, you cannot simply do whatever you want on my system with your software. That's ... sort of the whole point of a software distribution, in the Linux world!
When I want the latest version of a CLI tool on Linux, I simply `pacman -S package`. That's it; one command. I don't see how it could be any simpler or better than that, and on top of that I'm getting the benefits of moderation and integration with the rest of my system. Perhaps you are emphasizing latest version here, and hinting that you don't get that on Linux distros? That depends entirely on the distro; a software distribution is (roughly) a collection of user interests. An Arch user wants (and gets) the latest versions of all upstream software. A Debian user does not want this or see constant updating to the latest version as an advantage, so that's not what they get.
My understanding was Arch versions often lag behind that of the latest binaries published on GitHub (based on only mucking around with Arch once); however when I checked the AUR just now everything I use was up to date. Cool.
> A Debian user does not want this or see constant updating to the latest version as an advantage, so that's not what they get.
I'm not sure I agree. You can value Debian's stability and also want to install the latest versions of some tools; that's where I'm at, and package managers that just download statically linked binaries work for me nicely.
What? Rust can cross compile perfectly fine.
> C / C++ -- certainly possible, but you'll need on retainer a UNIX graybeard, or at least an autotools-foo master;
If you write C and don't know how to cross compile your code you shouldn't be writing it.
> Python -- you're already covered, just read the zipapp documentation;
Python actually can't cross build (reliably).
> Java -- possible, but you'll need a startup script to just call java -jar some-tool.jar; (also not a good fit for short-lived tools, mainly due to startup times;)
Use GraalVM.
Caxa - Package Node.js applications into executable binaries - https://github.com/leafac/caxa
It can cross-compile too.
It will track all uses of reflection during a program's execution. You can then use the resultant config as the starting point for the reflect-config.json.
Edit: it seems to me that GraalVM would benefit from a repository of classes and corresponding reflection configs. Like DefinitelyTyped from the Typescript world
But of course, your linker must support it. I don't know where to get a linker for Linux ARM in macOS. On Debian it's apt install.
Cross-compiling in C is always miserable to set up, especially around the area of libraries. And it gets worse if one of the compilers is from a different vendor.
I would like to add that Zig is extremely cross-compilable. You can even use it to cross compile C/C++ projects.
And Nim is fairly easy to cross compile. Just output C or C++ and cross compile that. I don't know why people are saying that cross compiling C or C++ is so miserable. Yes, it's not always a one-liner, but that depends on the toolchain. For example, there is crosstool-ng. Also Linux distros tend to have cross compilers for many architectures right on the box these days.
Here is how to cross-compile to 64-bit RISC-V on my Ubuntu machine:
sudo apt install gcc-10-riscv64-linux-gnu g++-10-riscv64-linux-gnu
sudo apt install gdb-multiarchYMMV; my experience was actually the opposite, approximately an year ago. I was actually really surprised of being able to build na executable for RISC-V (that I then copied to a VM, and run) all by adding 4 lines to Cargo's config.toml.
This statement is the programmer equivalent of manspreading.
I didn't mean to come off as elitist, but I have never met anyone I'd trust to write a C program that didn't have big obvious bugs that didn't know how to cross compile/statically link a C executable. If you haven't, that means you haven't spent enough time actually coding to be able to feel safe about what you're writing. Which is fine. But then use a language you know or spend more time learning about your tools before complaining. C is simple, everything around it is difficult.
It would be great if GCC supported all target architectures out of the box, but what would be needed for multiple OS/platform support?
GraalVM can't cross-compile. If you want to compile it for a particular platform, it must be compiled on that platform.
pyinstaller --onefile will create a signle file executable with an embedded python runtime and libraries (no need to install python) https://pyinstaller.readthedocs.io/en/stable/usage.html
Another thing to add, if you use run pyinstaller in docker with an old base image and copy the executable out, you can make a pyinstaller executable that will run on a vast range of machines due to runtime backward compatibility. I have single executable python apps that run fine on redhat, debian, ubuntu and suse.
Can I *easily* compile a non-trivial C/C++ application from say Linux to OSX or OpenBSD, for simplicity say all of them are x86 64bit? (Bonus point if the application in question happens to use LMDB or another non-trivial library that plugs intimately into the OS API.)
About Rust, I've once tried to compile from Linux to OSX a quite trivial application (basically a glorified `md5sum`), and the experience wasn't pleasant, and it involved lots of "internet searching" and basically stumbling around... (I don't remember if I succeeded, but I definitely wouldn't try it again.)
I don't even consider that to rise to the level of "cross compiling".
Getting started with emscripten to target WASM for C and C++ is rather a chore of dependency wrangling IME. Targeting WASM from Rust, OTOH, is trivial. Targeting windows from linux with Rust is also quite straightforward, as has been experimenting with targeting consoles or Android from Windows.
Targeting a MIPS32 OpenDingux target from Windows was much more of a chore. The toolchain with libs, headers, etc. that I used is just a *.tar.bz2 that expects to be extracted to /opt/gcw0-toolchain of a linux distro specifically, and embedded absolute paths all over the place make changing that difficult. I do resort to WSL on Windows, basically only because of those embedded paths: https://github.com/MaulingMonkey/rust-opendingux-test
Acquiring the appropriate libs and headers to link/compile against for cross compiling is always an adventure, but Rust isn't making things any worse IME.
===========================================
no dedupe: 16.0 GB (17228518418 B)
dedupe: 11.7 GB (12594871766 B)
singlelet: 8.1 GB (8677649129 B)
orphan: 1.8 GB (1889889290 B)
===========================================
deduplicated size ratio: 73.10
singlelet space usage: 68.90
singlelet file ratio: 63.56
orphan space usage: 15.01
orphan file ratio: 2.63> C/C++: certainly possible, but you'll need on retainer a UNIX graybeard, or at least an autotools-foo master
I don't understand this. The "C/C++ is black magic!!" meme just never seems to go away.
They both support static linking and always have. It mostly comes down whether you dependencies will work with it, and setting appropriate compiler flags. In other words, not terribly different than anything other supporting language.
Also why would you be using autotools in 2022? CMake has been the de facto standard for about a decade now.
If the app has assets it gets a little crazy quite quickly.
I always wish that’d taken off.
I guess AppImage isn’t dissimilar in that it appends a squashfs image to be mounted with FUSE.
[0] Which of course came from NeXT Application Bundles, which is also where OSX Application Bundles came from.
It's true, though. Proof of that is how many tools are made basically because no one wants to deal with Makefiles and autotools. I'd argue that even Docker fits this category of tools made to hide C/C++ builds away
no one wants to deal with Makefiles and autotools because as of 2022 they don't support spaces in paths and the most used desktop OS on the planet happens to have spaces in the default path for installing software
The dominant libc for Linux, glibc, says "static linking is highly discouraged", see [1] among many other sources for the subtle problems it causes.
There are solutions like [2], [3], and [4] but they all have downsides and don't compare at all to e.g., the go (or even rust) single-executable story.
---
[1] https://stackoverflow.com/questions/57476533/why-is-statical...
[2] Link glibc anyway and try to avoid the pitfalls.
[3] Link glibc dynamically but everything else static, and build your exe using an ancient toolchain (or better, a new toolchain built for an ancient libc) so that it works on most distros.
[4] Give up on glibc and statically link an alternative libc like musl.
But yeah that's pretty rare tbh.
To work around this you need to try to build on an old of a distro as possible, so you use only old symbols... but you don't want to build on an ancient compiler since it might not support the language version or compiler flags you need, so you first build a new old toolchain on the old distro against the old libc, or find a docker container from someone who has already gone though this pain.
Of course, this also means you can't use new libc features: libc is more than just the C runtime, it's also the shim layer between userspace and the kernel, so you might miss out on newer system calls (or have to write the wrappers manually, etc).
It gets even worse for C++.
> [3] Link glibc dynamically but everything else static
Go does this. Zig links musl [4] by default but gives you the option to link glibc. I don't know about Rust.
I don't think so: they only make it ABI backwards compatible (compile against old glibc, run against new) and API compatible (recompile your source against different versions should work).
I just checked by glibc and it had > 200 symbols versioned 2.34 which is the current glibc version. If you use any of those you can't run against older glibc.
Ok, I can see what I have written that could read this way, but the context is completely about backward compatibility. Forward compatibility isn't a concept that appears often in software.
E.g., you might use a relatively recent distro in your GitHub action (or whatever) to build this binary, but then anyone running CentOS or whatever can't run it because they ship an older libc version and required symbols aren't available.
You seem to want some way to have your code run on any platform whatever features you use in it. That you can't have. Of course if you want to support CentOS 6, you have to avoid any new symbol.
Combine that with the multiplicity of build systems, and it's easier to just use dynamic linking. Go and Rust solve that by having a standard build system.
The result:
C++ source pushed to git => GitHub pipelines => freshly compiled binaries for all platforms
That said, I have also re-written some things in Go just for its ease of packaging and cross-compilation. The fact that libraries "just work" is like a Zen experience to me. Also, "go generate" really makes it easy to merge small resource files into your one-file binary app. BTW, I come from python and Ruby, where "WTF did mkmf.rb crash?" is a daily thing...
it's not possible for GUI apps as you would have to link against the OpenGL drivers statically (and thus your software would only work on one brand of graphics card)
> docker run just-another-2g-vm-image-running-a-fully-fledged-os-with-all-the-systemd-glory-masked-as-a-container;
Docker doesn't run a "vm" image and it definitely doesn't run another kernel (the kernel isn't included in docker images because it's designed to run on the host kernel). It's more closer to chrooted process (if you just consider the process) than anything... combined with some resource isolation.
> that certainly won't lose any data when you purge it by mistake
If you don't understand what you are doing, you can "purge data by mistake" doing other things as well.
Docker images come with ALL the pros listed in the article with fewer cons. For example, they are isolated from the get go by design, so fewer security concerns than running a binary on your system a your user without namespace isolation (namespace isolation is what's happening with docker). You can choose what files and devices process running in docker has access to.
The download size may be bigger than a statically linked single executable... but if you are running a lot of different processes in docker, they may share the underlying layers which may mean that you end up using less disk space overall.
Yes I do understand them!
In fact, in a former life (i.e. during the long forgotten time when Linux containers were just a novelty and LXC was seen as the "future") I was a big proponent of containers.
What I was not a big supporter of is containers, that instead of being as you well say "more closer to chrooted processes", instead they are almost stand-alone VM images lacking a kernel. In fact Fly.io does exactly this -- they take a Docker container, add a Linux kernel, and run everything in a Firecracker "micro" VM.
----
> that certainly won't lose any data when you purge it by mistake > If you don't understand what you are doing, you can "purge data by mistake" doing other things as well.
Yes, butter-fingers are a thing... But with Docker it's so easy to delete a container by mistake (that although shouldn't have state it does); on the other side, if you are about to issue `rm -R -f .` and you are inside `/var/lib/mysql` you do have quite a few opportunities that perhaps you are doing something stupid (you have to explicitly write `rm`, add `-R` and manually type `/var/lib/mysql`)...
----
> The download size may be bigger than a statically linked single executable... but if you are running a lot of different processes in docker, they may share the underlying layers which may mean that you end up using less disk space overall.
OK, say that for "server applications" (that happen to use tens of micro-services) this makes sense.
But how about a "tool application", say a static-site builder; does it now make sense to use Docker to run this tool?
Regarding using a container for distributing static site generators, does anyone really do that? I think you may be building up a straw man here, I haven't seen anyone recommend this workflow. Could you elaborate on how this relates to the parent comment's mention of using less space when running many containers? What the parent comment mentions is, in fact, highly relevant to some of the main use-cases of containers, which is running many containers of the same service on one or many hosts. Space can be saved there with layer caching.
Is Sphinx a good enough example? <https://www.sphinx-doc.org/en/master/usage/installation.html...>
----
> Could you elaborate on how this relates to the parent comment's mention of using less space when running many containers?
Could you elaborate on how does "running many containers" relates to the topic of distributing and running *a single tool* (that was the context of the article)?
YOU may be distributing just a single tool but your users most likely use the computer for more than one tool. If they use many tools under docker, it is likely that many of those tools share the same underlying layers.
How are docker images even close to a VM image? Docker image is closer to shipping your application in a zip file with all the dependencies than it is to a VM image. In a VM image, you need a "minimal OS" which has to include the kernel, drivers, required OS binaries + everything that would be in a container.
Docker processes are literally running natively on host machine making linux system calls just like any native executable would... except they only see the resources you want them to see. Docker processes don't need the docker daemon to be running. In contrast, if you were to run a VM, you'd have to have at least minimal OS kernel with all the required OS processes and drivers and the application on top of that... not to mention the hypervisor itself.
> But with Docker it's so easy to delete a container by mistake
You do have to write "docker rm" to remove a container. It doesn't just disappear. The only way I can imagine someone losing data is if they think they are writing to the host filesystem when inside a container (without using volumes as one should) and delete the container thinking that their data exists in the user filesystem.
> But how about a "tool application", say a static-site builder; does it now make sense to use Docker to run this tool?
This is how you can run a "tool application":
docker --pull run --network none --rm -it -v $(pwd):/src klakegg/hugo:latest
A docker image and a VM image can be the same.
For example say you have a simple web server that just listens on port 443 and does it's thing; if you compile it statically, and it doesn't need any other files, you don't even need a VM image.
You can simply create an initramfs that contains that executable (nothing else, no drivers, no configuration files, no distribution, nothing).
Just boot the Linux kernel (granted, a custom build that uses built-in modules that the hypervisor actually uses), use proper kernel arguments to either statically or dynamically configure the network stack, and instruct the kernel to use the server executable instead of the `init`. Your server doesn't even know it's running in an "empty" VM.
(Just read about how AWS Lambda implements everything by using Firecracker.)
I'll add to that READMEs, LICENSEs, SBOMs (Software Bill of Materials), example configuration files, etc. How to supply all those files when all one gets is a single binary executable?
Simple! Bundle everything in the executable.
As a bonus, because the tool outputs these files, it can now generated them dynamically. For example instead of a bland configuration file, with all the possible integrations commented out, it could either try to auto-detect where it's running and what's available, or present the user with a question-answer session to fill in the details.
----
For example, a pet project of mine <https://github.com/volution/z-run>:
z-run --readme # shows the README with `less` (if on TTY) or to `stdout`
z-run --readme-html # for the HTML version to be opened in `lynx`
z-run --manual # or --manual-man or --manual-html
z-run --sbom # or --sbom-json or --sbom-html
It even gives you the source code:
z-run --sources-cpio | cpio -t
----
So, does your tool need a `.desktop` file? Just create a flag for that.
Or, if there are too many such extra files needed to be placed wherever, just provide an `--extras-cpio` and dump them as an archive. Or if placing them requires some work, provide an `--extras-install`, but before `sudo`, kindly ask the user for permission.
Granted all this requires some extra work, and increases the bulkiness of the executable, but:
* all that extra code can be extracted into a common library; (I intend to do that for my software;)
* if all these are compressed, especially being text-only, they are a fraction of the final executable;
----
I am especially proud of the `--sources-cpio` option. Is something broken with a particular version of the tool that you rely on? Great, instead of bumbling around GitHub to find the particular commit that was used to build this particular version, I can just get the sources from my tool and use those. All I need is the build tools, which in case of Go is another `.tar.gz`.
I want to highlight a few ideas that perhaps I didn't cover very well in the article:
* static linking -- the concept of static linking appears in total 7 times in the article, once in the summary, once qualified with "if possible", three times as comments on a few example projects that do happen to employ it, and once discussing Go's advantage; the article is not about static linking!
* dynamic liking -- it is completely fine, just don't depend on non-mainstream libraries, and don't require the version released yesterday as most stable distributions won't have it (for the next couple of years);
* packaging and dependency minimalism -- as I've noted on Lobsters (<https://lobste.rs/s/adr60v/single_binary_executable_packages...>), the article is mainly about the simplicity of building, packaging, publishing, deploying; try to read it both from the point of view of the developer and the user;
I think you could add an entry to the "Languages suitable for single binary executables" section; these days, .NET (C#, F#) single-file applications work very well. This is a fairly recent development (single-file support has been good for ~1.5 years now) but I've been very satisfied. https://docs.microsoft.com/en-us/dotnet/core/deploying/singl...
Technologies to look at:
* Warp Packer -- https://github.com/dgiagio/warp/
* Liberica Native Image Kit -- https://bell-sw.com/pages/liberica-native-image-kit/
Warp Packer bundles my JavaFX desktop application, KeenWrite, into single binary executable files:
* https://github.com/DaveJarvis/keenwrite/releases/download/2.... (Linux)
* https://github.com/DaveJarvis/keenwrite/releases/download/2.... (Windows)
* https://github.com/DaveJarvis/keenwrite/releases/download/2.... (JVM)
The start-up time for the first launch of the .bin or .exe is slow because it unpacks to a user directory. Subsequent starts are fast enough, even when running from the command-line as a short-lived task. Here's the script that creates the self-contained executable files:
https://github.com/DaveJarvis/keenwrite/blob/master/installe...
To create a release for all three files, I run a single shell script from a build machine:
https://github.com/DaveJarvis/keenwrite/blob/master/release....
I could probably generate a binary for MacOS, but not enough people have asked.
2. Being able to patch dependencies is important. This isn't so much a feature to help developers, but one to help users. You could technically do this without dynamic linking, but have to make it easy to recompile a project with an updated dependency (which most package managers don't). Not being able to patch a package's dependencies puts the developer accountable for your system administration issues, which is not scalable.
These days, if you publish a single-file app all .NET libraries live in the executable and do not need to be unpacked at runtime. Native libraries still need to get unpacked at runtime (if your application uses any; many don't), but you just set the IncludeNativeLibrariesForSelfExtract build property and everything else happens automatically for you.
I'm working on getting IncludeNativeLibrariesForSelfExtract set by default, upvote here if you think that's a good idea: https://github.com/dotnet/sdk/issues/24181
Last shop I worked at was ... well, they were of an age and came from an era when installation meant "drop some files in a directory, set up an icon on a desktop, what do you mean 'dependencies' and 'framework' and 'installer'?" I did my best to accommodate, because that seems more reasonable to me, but it certainly wasn't simple.
I do not dislike the concept of the Registry, it is an interesting tradeoff against the skillion dialects of .ini files and other configuration, but much further than that, it feels like installation of software is akin to a drop of ink in the water -- you can't really get it back out again.
My impression is that OpenSUSE go packaging is a bit behind though. But I'm not an active packager so this may have been fixed.
I know there are many arguments to support this style, but I simply dislike it from an aesthetic point of view, too messy.
(with some limitations, but the .NET stuff with .NET dependencies including the entire .NET Runtime should at least be in there, and you can compile resources like icons or other assorted data embedded into the executable as well)
Unfortunately it would also would be really good for viruses.
■ curl -O http://harmful.tool/all-source-code.gz
■ Audit ALL of the source code
■ Build the "reproducible build" in a sandbox
■ Test / Run in a sandbox
■ Delete!You are knocking wrong door. Try SAAS instead for your critic.
This is ironic. Do people really version their binaries in git repos?
IMO it is no problem if your software is several hundred megabytes in size, I have 8 TB of hard drive. Storage is cheap as long as you aren't using Apple.
I mean, very often the Tensorflow version a piece of software wants doesn't actually have a version available for my currently-installed CUDA version. I'll have CUDA 10.1, 11.0, and 12.6 already on my system for other pieces of software, and your software is hellbent dependent on Tensorflow 2.3 and CUDA 11.1 and I'll end up having to install 11.1 in addition anyway just to run your software.
I could probably get similar benefits from learning, say, dpkg and systemd really, really well... but then that'd only work on dpkg-using Linux distros. Meanwhile, docker commands, docker-based shell scripts, and docker-compose files work just about identically ~everywhere, and can even work nearly-transparently on systems that don't actually support linux containers, like macOS and Windows, via virtualization and some command shims.
I hardly even care that it uses containers. The parts of it I like don't actually require that—though they do encourage people packaging for Docker to document things well and to cut through a certain amount of various packages' config bullshit. Take Samba—it's like 100x easier to configure & manage for common use cases using Docker than most distros' default packages & config files. There's no actual reason that needs to be the case, it's just that the process of containerizing it brought that as a side-effect, but that's one of the main reasons I'm using Docker, not the containerization per se.
Wsl2 relies on a hypervisor (i.e. it has to run a Linux VM underneath) so running docker apps on Windows comes with all kinds of problems from poor performance to file permissions being messed up.
A mature wsl1 would be able to run almost any Linux app on Windows natively.
The old Delphi-based Skype for Windows was a single executable. At least, most of it was in that big .exe.
Userspace Linux is different in that not only are these libraries typically installed separately, they all have varying degrees of compatibility between versions.
Also, docker seems to be on the way towards non free (or already in a?) /non-foss , so will probably soon be completely out of favor with everyone. Perhaps other tools like jails,bhyve, etc will rule the space soon.
Of those users 90% are running Debian. So you insert apt install docker into your script. The remaining .4% are using red hat. So you insert yum install docker into your bash script.
Then you add a line to start your image, now put the bash script on your website, voila, single file program that works anywhere.
Once you have root just do whatever you want. replace the binary, install another version, upgrade the kernel, and then run your app.
Most users here using the software may not understand anything about software dev, so they need it to work on a laptop with an external hard drive - no cloud compute or cloud storage (to keep cost low for them) - but it can't have any long lists of commands to get something working.
One command to install and run docker seems to be a decent solution, but the program to do that needs to be accessible from any OS without gaurantee of docker being installed.
With the added hassle of installing Docker and learning and managing all of its details, and what you get in the end only works on Linux.
Meanwhile every OS in existence support running its executable format with zero other bookkeeping.
On Linux the only reasonable solution is an install/executable script and/or docker.
Just as long as they're signed. Unsigned apps built on one machine will fail to start (pass audit) on another.
The blob might be a statically-linked executable, a flatpak, a docker container and so on.
All this methods go against OS-based packages and prevent end users to update vulnerable libraries.
It is well proven that things like containers on docker hub age like fish, leaving many unpatched images around.
This is why dynamic linking and Linux distributions exist! Stop reinventing the wheel in worse way.
2) Network bandwidth for updates is far from abundant for less wealthy 4 billion people on the planet.
3) Same for drive space. A lot of phones/SBC/IoT devices have limited storage.
4) Same for I/O bandwidth. (E.g. even on a new PinePhone bulky applications take many seconds to start.)