Is it actually possible to do hot code reloading while preserving the state of the running system in Common Lisp? I'm genuinely curious, I kind of had this impression that it was a feature fairly unique to Erlang.
Yes.
> I'm genuinely curious, I kind of had this impression that it was a feature fairly unique to Erlang.
It's not. iIRC there's even a version of the JVM with code reloading.
What's interesting about Common Lisp is that not only can code be dynamically compiled into the running image but class definitions can be changed and live instances will be updated without stopping the program or reloading anything. There are a tonne of features in Common Lisp like this that are geared towards robust, maintainable software.
Yes, because of Java's extensible classloading mechanism, all you have to do is drop an war/ear to the 'deploy' folder of your application server. New files are deployed, rewritten files are redeployed and deleted files are undeployed.
One of the beauties of Erlang's hot code loading is the ability to migrate internal application state. An example:
I'm storing a lookup table going from key to value as a tuple {key, value}. With a change going out, I'm going to start doing {key, {value, other-value}}. In Erlang there's an easy pattern in place to do that. Just provide a function that goes from {key, value} to {key, {value, sensible-default}} and as part of the code reloading process that function will be run before the new code starts executing.
That's not at all a perfect explanation of the process, but it's more or less useful enough to describe the difference between the two platforms.
if you need more elaborate lifecycle management of components as they are upgraded, then maybe OSGI could do that.
erlang's approach really appeals to me though -- not only does the code get upgraded, but it's in cooperation with its message loop so you don't miss a step, and you also get a callback to migrate your actor state.
I've seen some info about the JVM implementation, but I could've sworn it wasn't able to preserve state while hot swapping. I haven't dug in to it though, so it's totally possible it handles that.
> There are a tonne of features in Common Lisp like this that are geared towards robust, maintainable software.
I'm a fan of Lisp in general, though my experience is limited to a tiny bit of Scheme and experimenting with Clojure. I'm glad to see that stuff like this is available in at least one of the Lisp variants! Question for you (if you program in Common Lisp): What are the main selling points? I spend a lot of time already with a couple of languages, Elixir/Erlang (for fun), C#/F# (for work) - what is it that would make me want to switch from one of those to Common Lisp?
Thanks!
Erlang has them in the form of parse transforms and Elixir has them too and makes them easy to use. Not as easy as sexp based languages, but almost.
1. Configurable compiler. You can tell the compiler where it's safe to ignore types and where to be strict about types. For the battle-hardened critical sections of code turning off run-time type checking can be a pretty big win performance-wise.
2. Extensible compiler. Macros are like tiny compilers. It's possible to create DSLs for the programmer to work with a problem in terms of nouns and verbs that make sense for their task while still compiling to VHDL for your application. Common Lisp becomes the substrate on which you build the language you need to solve your problem.
3. The best error handling system I've seen in any language. Signals, handlers, and restarts allow higher-level code to choose an appropriate strategy for recovering from errors under various conditions without losing state and restarting the computation.
4. Top-class introspection tools. The JVM has been able to do some of this stuff for a while but with CLOS and the error handling facilities available Common Lisp makes this so much better. You can inspect any object, change values on slots, trace any function, step through the entire stack, and recompile code into the running image. There are nice tools written on top of these facilities which provide some of the most elegant debugging sessions I have ever encountered. Stop your program? Why? I can connect to a running image, inspect the running program interactively, and compile the fix into it without restarting. Even over SSH directly from my IDE.
And that's just the stuff in the specification. I'm a fan of the newer ASDF systems for defining systems and their dependencies (implicit file-system structure for defining module composition? No thank you). There are some really solid libraries available (iolib, bordeaux-threads) and a very nice FFI for interfacing with alien code. And various implementations provide interesting extensions depending on what you need. Clozure Common Lisp, for example, has really good native Cocoa bindings. SBCL has a very slick compiler for generating fast code. Most of them provide threading extensions and there are libraries which abstract away the differences if you're writing a library that depends on threads.
For example, how well CL's class redefintion and instance reinitialization work with preemptive threads? It isn't obvious how each implementation address this issue by skimming docs, so I'm curious if anybody with direct experience. When I implemented CLOS/MOP style OO in my Scheme, I ended up using a global lock during class redefinition, for it need to modify lots of places. It's MT-safe now, but I'm still wary to run class redefinition on a system written in my Scheme, which is busy handling requests.