Debugging Lisp Part 3: Redefining Classes

I believe you are misunderstanding what I did. At no point did I change an object's type or change its behavior. All I did was change the object's implementation in order to make it more efficient. Being able to redefine the class effectively allowed me to JIT compile the code by hand.

It feels similar to database migrations: sure, you can change the schema, but if the code is changed accordingly at the same time, the migration will happen smoothly, without having downtime.

The fact that other languages don't feature restartless updates is as strange, from a point of view, as it is to imagine using a database that forces you to have downtime.

Of course, we go around that by having multiple servers that restart in sequence… but to think that people were able to debug and update spacecraft software while it was running, without downtime, is pretty impressive (the story is narrated here: http://flownet.com/gat/jpl-lisp.html).

I don't think anyone would redefine classes as part of the normal operation of a running production system. At most it'd happen during the initial startup.

The obvious case where you'd do it though is during development. Development in CL is generally image based. You start a new Lisp instance, load up the code once, and then just modify the running system by redefining functions or, indeed, classes.

There's a couple of exceptions. One is upgrading a running system without taking it out of service. That's less tempting these days when everything is distributed anyway, but might be relevant for very long running batch jobs that run into some kind of problem halfway through the problem. The other are systems which are effectively configured through code.

And of course class redefinition is a pretty vanilla dynamic language feature. The only bit where CL is special is having a well defined protocol for upgrading "old" objects to "new" ones.

> the solution given seems overly complicated for the problem at hand

The Hello, world program also seems kind of complicated for what it does. Who wants a program that prints a greeting on stdout and exits? Best to regard this problem as the simplest possible one to demonstrate the mechanism. A problem that really warrants it would be far too complicated to present without going into a lot of irrelevancies.

> redefining classes at runtime seems like a really bad idea

My experience with class redefinition comes from Smalltalk, not Common Lisp, but I am pretty sure they are somewhat similar. You really need to have experience working with an image-based language as an image-based language [1] to understand why such functionality can be useful. Additionally, I think you'll find that class redefinition is not intended to be something that would ever make it into production. Regard it the same way you might regard a debugger's feature that lets you change the value of a variable when at a breakpoint then resume execution: something that lets you continue with the invocation of a program you have a lot time and effort invested in, say hours worth of computation building up internal data structures, which you would lose if you had to make a one-line change, recompile, and start the execution again from scratch. It's a powerful piece of functionality that should probably be used sparingly, but when you want it, you really want it.

[1] I.e. not frequently recompiling from scratch and invoking afresh each time, but just updating those bits of code you've changed, and keeping the data already built.

You have a production server that uses an ORM. You push some change that change the database table. The class redefinition code updates all instances of database objects, and the ORM kicks off an automatic migration[0] so the database and the objects are in sync.

[0]: e.g., using Crane http://eudoxia.me/crane/

It seems to be a bad idea until you realize that there is a protocol that takes care of class redefinitions without breaking everything. It is still possible that when changing your class, you introduce a bug in your application, in the same way that it would introduce a bug if you stopped your C# program, modified a class and reran it without update some other part of the code that did not adjust its assumptions.

OP mentions updating a running service, but I think I can be wise to suggest doing it first on a test machine. The kind of bugs introduced are however not due to a broken understanding of classes by Common Lisp, but rather on the consequences of the changes in application code. I don't see how "this just leads to undefined behaviour within your type system", though.

> If I pass an int to a method I don't expect it's type to be changed by said method.

I made a little game where objects could go beyond the limits of the world. In the UPDATE method of those classes, when the object escapes the boundaries of the world, I would change their class to the empty GARBAGE class. In a later pass, I remove all instances of GARBAGE. This is equivalent to having an ALIVE? member in all objects, except you don't need to.

When developping the game, I would periodically discover that my classes would need to be different that what I originally thought, and I modified them in the current Lisp environment instead of stopping the program and recompiling it. This is very convenient. Of course, changing classes at runtime becomes rare when the programs is run by the user of your program.

> In C# if I pass an int to a method that can't handle a following int*int because the result will overflow 32bits, then I'll just do a '.. new BigInt(int)' and go from there.

Why are you only talking about ints? The result would not overflow in Common Lisp, it would return a bignum. Changing a method at runtime is useful for live-coding, exactly as for standard functions.

Now, look at "cl-protobufs": you define a Lisp system (defsystem) where you add a depedence to a :protobuf-file, and here is what you get: the .proto file is parsed as a protocol buffer schema and generates a set of classes and methods in a new package. You can do this in java with Maven, of course, but here you generate code within your language, without restarting your environment.

> Probably going to get downvoted to oblivion but redefining classes at runtime seems like a really bad idea - surely this just leads to undefined behaviour within your type system.

Technically, databases are doing that all the time - at least the more advanced ones like Firebird and PostgreSQL - you can transactionally change the schema, even while the DBMS is running, and you do not expect to lose old data in the process. The data gets even updated lazily. Interestingly, there's at least one database (AllegroCache) that uses this similarity between heap update and on-disk data update to great advantage to blur the distinction between transient and permanent data further still.

> If I pass an int to a method I don't expect it's type to be changed by said method.

"Ints" are immutable, largely even on a meta-level. Most of them (fixnums) in any program are even direct (not pointers), so no mutation would affect any callers as a side effect anyway even if it were possible. At best you can redefine methods specialized on numeric classes.

But I'm still learning these things, I hope I'm not spouting some nonsense here.

As a part of production code, I agree, redefining a class is a bad idea. However it absolutely shines during development. I am working as a professional lisp programmer, and it is a very common case that I want to add a slot to one of my classes. Being able to do that without having to restart the software in development is very nice as you are not distracted in your programming work by the distraction of restarting.

If you're updating a class then you're either (a) relying on existing functionality being maintained, or (b) changing functions that use instances of that class.

An `int` is likely a pathological example, btw; you most often work on classes that exist in your problem domain, like entities. It doesn't (generally) lead to problems in your type system because updates include both classes and the functions that operate on them.