Self_update: In-place updates for Rust executables
github.com
github.com
As others have pointed out, it's not a perfect solution and has a limited scope of use. I originally made this as an experiment to make it easy to distribute updates for cli apps that I was working on. Yes, a package manager is the "right" way to do this, but since I was testing and building images in my CI flow (using https://github.com/japaric/trust) I wanted a way to take advantage of that pipeline in a seamless way. The library exposes a couple predefined "backends" that handle common patterns like GitHub releases and s3, but the underlying steps (downloading, extracting, replacing) are also made available so you can build your own update pattern and add things like verification.
Nice, through not production ready yet.
Through I'm a bit hard pressed to find any good use-case for it. In nosy installations the binary _should_ not have the right to override any binaries at all, including itself. In Linux (and I think windows too by know) there is a form of update functionality provided by the system. Even if you do your own update for client side software it should be a separate binary (as the main binary should not be able to so so). Maybe it would be ok in a container, but the restarting the container would likely instead the update.
(As a side note I said should because I know that it's sometimes not the case, at least not without putting additional work into it. Still using a update mechanism like that would make it harder to make it the case in the future).
Why would you want to do this without restarting the process? Isn't hot-reload a much more desirable thing?
and, to that end, replacing a binary in place just ensures that what's running cannot be re-run on stop, so you could end up in a situation where you've overwritten a working binary with a nonworking one which could obfuscate critical information when trying to recover service.
This is true of any self contained updater.
I'm sure everyone believes that hot reload is the answer, there are probably fewer than 5 projects where it has been done, and all those projects likely have some seriously PHD-level code.
Beyond the most extreme manual and cognitive effort, hot reload is completely and utterly unsolved.
The fundamental problem with hot reload is that data structures can change. If the old process has a priority queue implemented as a linked list and the new process has a priority queue implemented as a heap, how on earth is any automated mechanism going to copy the state across?
If you want to change the database schema without losing data, you need to write a migration. If you want to hot-reload code with different data-structure definitions, you'll have to write a migration.
A hot-reloading solution for a statically typed programming language will at minimum have to detect whether data structures have changed and either prevent the reload or require that the new code includes a function to migrate the data.
http://www.flownet.com/gat/jpl-lisp.html
> The Remote Agent software, running on a custom port of Harlequin Common Lisp, flew aboard Deep Space 1 (DS1), the first mission of NASA's New Millennium program. Remote Agent controlled DS1 for two days in May of 1999. During that time we were able to debug and fix a race condition that had not shown up during ground testing. (Debugging a program running on a $100M piece of hardware that is 100 million miles away is an interesting experience. Having a read-eval-print loop running on the spacecraft proved invaluable in finding and fixing the problem. The story of the Remote Agent bug is an interesting one in and of itself.)
That would be a quite tricky thing to do. You would probably need some very specific architecture in your application to allow for something like that.
It's not clear to me that this is a better architecture than just splitting your application into two or more processes, one of which is of such lasting value and high quality that it would never need replacing. Then still, you may be better off making your gateways/endpoints not need 100% uptime in order to maintain availability, or making your client applications not require total availability to behave well.
In case of rust batteries aren't included so each network stack will require different strategies, but in general it will be more like nginx (or unicorn if you come from ruby world) style of reload.
In other words - it's possible, but why?
Furthermore, channels aren't the primitive used for multiplexing IO / handling connections on a socket in go. You typically have a goroutine (e.g. 'http.ListenAndServe' spins up goroutines), and the gorutines are managed not by channels, but by the internal go runtime's scheduler and IO implementation (which internally uses epoll).
Because of all those things, replacing a running go process that's listening on sockets is no different from that same problem in C. You end up using SO_REUSEPORT and then passing the file-descriptors to the new process and converting them back into listeners. Channels don't end up factoring into it meaningfully.
If you're interested in what this looks like, cloudflare wrote a library called tableflip [0] which does this. I also forked that library [1] to handle file-descriptor handoff in a more generic way, so I've ended up digging pretty deeply into the details of how this works in go.
You could have your program use SO_REUSEPORT (or SO_REUSEPORT_LB?) on the socket it listens on. When upgrading, the old version launches the new version, which begins accepting new requests on the configured addresses and ports. The old version closes its listening socket so all new connections are handled by the new version. The old version can then just wait for all existing clients to disconnect, or if the server handles long-poll style loads, maybe send a redirect of some kind to force the client to connect to the new instance. When clients are all exited or some acceptable timeout has expired, the old version can delete itself and exit.