C and C++ Hot-Reload/Live Coding
liveplusplus.tech
liveplusplus.tech
I'm using Cling, as part of jank, a native Clojure dialect on LLVM, with C++ interop. This allows me to have truly interactive, dynamic programming, while still reaching into native code. jank source compiles to C++, which is then JIT compiled by Cling. In recent benchmarks, it can be several times faster than Clojure JVM. https://jank-lang.org/
Ultimately, this isn't something Cling needs to worry about. You can have your cake and eat it, too. If that sounds too good to be true, know that it's how Clojure devs write all programs. We spin up the process and then send code to it from our editors, without needing to tear it down.
jank gives you this, in C++ land, including support for inline C++ within your Clojure code. I'll have a lot more demos coming out this year, but you can also check out the jank blog or my talks at ROOT's compiler research group and Clojure Conj 2023.
The problem is that too many people insist in using it like C.
Is as if one changes the JavaScript file extension to Typescript, and then keeps writing JavaScript ES3 on it.
>you can basically use inline C++ in python REPLs
That's exactly what I was getting at. There's no need for a pure C++ interactive environment. Just use python at this point.
And if that's in a familiar language, then even better. Clojure is a good choice, but I am still learning it.
Note: I am a lone programmer building Quant Engineering subsystems for personal purposes. So evaluate (pun intended :D) my opinion in that context.
How so? My experience is exactly the opposite.
Still, there are rough edges,
1. Rough: No (ergonomic) expression-evaluation available at the moment, hopefully tree-sitter based tools will solve this in the future.
2. Rougher: In the general case, I can't redefine a function/class. The 'global' namespace can be used to 'some extent' to achieve 'deterministic redefinition', but the code strays away from idiomatic Python.
3. Roughest: I have found Python performance to always be a concern. Maybe I am biased towards C/C++/Rust for performance purposes, aka Optimal Resource Utilization.
https://devblogs.microsoft.com/cppblog/cpp20-support-comes-t...
He dicusses techniques for this. one of which is a "migration" approach, similar to database migrations. Where, for example, you have a version number in your struct. and a function for migrating from an old version to a new version. I don't think he implements it in that video though.
Video for how it works: https://www.youtube.com/watch?v=pjGngeKgni8
To migrate from one class version to another, I found the easiest way was to provide a callback method in the constructor where one could save the state from the old class, and read it into the new class. For example: https://github.com/jheruty/hscpp/blob/master/examples/simple...
Ultimately though the limitations of hot-swapping this way limits its usefulness. My understanding is that liveplusplus is much more full-featured.
But I guess that’d require hooks into the compiler frontend, which is not just goes against your goal of being relatively compiler-agnostic but is also fantastically difficult for C++. (I wonder if DWARF contains enough data to at least transfer over identically-named fields?.. It’s also something of a bear, but nowhere near a full C++ frontend.) I don’t know how this could be useful to you, sorry. Thank you for showing your project, I’m grateful for the food for thought in any case.
Now, can we just all take a moment to appreciate that you needed a Smalltalk-style become:[1] and ended up making an object table, precisely the classic Smalltalk way? In the end it’s either that or precise pointer tracing, there isn’t (AFAIK) a lot of choice, but it’s still amusing to see an ostensibly high-level mechanism reproduced in a very different environment.
[1] https://gbracha.blogspot.com/2009/07/miracle-of-become.html
Do you mean like in Lisp with update-instance-with-different-class for example?
https://www.lispworks.com/documentation/HyperSpec/Body/f_upd...
When people ask why I don't like copilot and want a worse alternative when copilot "pays for itself", it's not because I don't want to pay, I just want to buy something and have it.
There’s replit-code[1], which is pretty much that I think.
You are buying time. Never pass up an opportunity to do that.
Maybe a few larger companies as well, because those subscriptions tend to come with support, and larger companies love to throw maintenance to non-employees. But if they cared enough about not having a subscription, they'd make it themselves.
The ones who don't benefit is individuals. I wish more products had one time costs for hobbyists (individuals with zero revenue) while having subscriptions for businesses and commercial. Lots of products have loopholes for education/OSS/etc so adding "hobbyist" doesn't seem like a big difference. And I think it would drive sales. If I use a product at home I'm likely to lobby for buying subscriptions at work.
Why do I want to spend hundreds or thousands of dollars for a perpetual license of photoshop when I hardly use it? A 30 day subscription once a year is perfect for me.
Even for those of us who are heavy users - can you see the game theory? The provider of your services has to deal with a huge amount of inertia in their product because of the continuous distribution model. The bigger the subscriber base, the more inertia there is. The consequence of this is that you are essentially guaranteed that impacting changes will be delivered slow enough that you or your business can adapt to them. Contrast this with a big bang software upgrade that requires explicit retraining and disruption to ops.
Not all subscription products are run this way, but the good ones are. Any product that has a 4 digit number in its name is a bad example. Visual Studio 2022 is not how a subscription model should go. Netflix is.
Made developing those scripts much, much more comfortable when it was introduce. You no longer needed to manually compile, stop and start the server, wait for all the database stuff to be loaded, switch to the client, log back in, wait and then retry on every single tiny change. To me, it was like magic.
Today, I wonder why they didn't just use a proper scripting language.
Still, RunUO was C# and like java, recompiling and loading is trivial. Doing it in C/C++ land is heroic.
Not to diminish the accomplishment or anything but is it that much easier in C# and Java?
It was always really easy to use the libloaderapi in Windows. Throw a vtable on it and baby, you got a stew going!
It's a sign of how bad developer tooling has been that live reload is sold as an add-on feature. DX seems always to see-saw between bill gate's c. 1990 demo of VB, and the compile-time speeds of a N' Wirth.
At a guess, when the internet happened, all DX had to be thrown away and we still arent back to where we were 30+ y ago.
I support being paid for your work, but live reload is a fairly trivial thing to write for yourself -- I spent one day this gone weekend writing a debugger in C which does it.
(consider also, https://github.com/tsoding/musializer which has live rebuild via the nobuild system in nob.c)
Update: there's a "deep dive" power-point that goes into a bit more detail: https://liveplusplus.tech/downloads/THQ_Nordic_Dev_Summit_20...
Can you expand on this?
Major Dropbox “who would pay for this over just using rsync?” vibes here. Never change, HN.
Or, more alarmingly, why are things like jrebel fawned over? Rather than, say, it being an embarrassment that java dx is worse than programming dx 30-40 years ago?
Why isn't the basic thing all OS do, ie., live reloading of code, a day-to-day essential element of all development?
I really don't know. But to be clear, it was 50+ yr ago, 40+ yr ago, 30+ yr ago --- and as of today, it's $100/yr subscription (lol?!)
I agree with your sentiment, but it's really hard to communicate it to the younger generations. I suppose it comes off as being negative, but it's really just melancholia over the loss of something that was fun, simple, and powerful.
consider how these architectures would be designed if must-have condition was sub-100ms iteration -- which should be trivial all-but-for codebases of at least 1mil+ lines; them, well, maybe it's 200ms.
This design isnt hard -- what's hard about save_frame(), save_state(), or w/e, and restore_state()?
Only that it would need to be actively maintained and developed -- that is, it would obligate developers to tend to their own ability to work.
What i think here is lost is not, somehow a simple system which can be magic, but the concern of developers to maintain such systems.
Why? Certainly, many reasons. But my comment here today is more to frame the issue: this 'feature' can, and should, be part of ordinary development. It isnt that hard.
Even if save_frame() is no longer a register copy, but somehow, an obscene json serialisation process.
Components depend on other components, and they all have to support hot reloading. And what about side effects? For example, an open network connection, or a window. You could store the state (handles etc.), but what about callbacks during the reloading phase? They'd have to block and wait.
Ultimately, what you need is a message passing architecture that supports all this. Smalltalk. Alan Kay has been a vocal critic of our modern architecture. See his talk "Alan Kay at OOPSLA 1997 - The computer revolution hasnt happened yet".
But that architecture is computationally inefficient.
I suspect the problem is that computers weren't fast enough for the last 30 years. So we focused on performant computing. We still do that, with high fps gaming. And adding better quality graphics. Soon we'll have Apple Vision Pro with a 4k display for each eye.
My personal hope is that WebAssembly and the Component Model will drastically change things, by introducing API virtualization and effectively decoupled modules, but this will require that tree-shaking/deadcode removal is disabled.
1. Come in, get coffee, check out latest code from Visual SourceSafe. 2. Hit F5 to start debugging. 3. Edit and Continue in Visual Studio 6.0 all day long. 4. Hit shift-F5 to stop debugging at the end of the day. 5. Go home.
Between that and the OG Pix for Xbox, the development experience has been all downhill from there. A lot of kids these days tried GDB once or twice and then grew up without even understanding the value of debuggers at all. And, by "kids" I mean "graduated 10 or less years ago" :P
I'm sure keeping that workflow working was a fair bit of work. But, it was worth it!
One of my university assignments, in the mid-1990's, was to implement a B-Tree library in C, which I used a OOP based approach, and already back then, gdb was powerful enough to use as a poor man's REPL.
I really don't get why people shy away to learn to use their tools properly.
If what yout talking about is "not that hard" can you show me how to do it?
If it's not that hard, why aren't you teaching anyone? Why isn't it ubiquitous?
Your comments seem so at odds with reality that I'm inclined to think your missing something. But I'm also intrigued. Is it not that hard? And if it's not hard, why is live plus plus successful?
exec() doesn't hot reload either.
Dynamic loading doesn't hot reload either, more like hot load.
Debuggers might hot-patch code to insert breakpoints, but that's not the same thing as replacing running code. Sure, the debugger might execute scripts you associate with breakpoints, but that's also not exactly the same thing as replacing the running code. Mind you, debuggers do have to make sure that such hot-patching is atomic, or they have to stop all threads.
Hot reloading has a ton of considerations:
- threading -- you need to quiesce all threads at points where they are not executing the code to be reloaded _or_ you need to load the new code and hot-patch either calls to old code and/or old code function entrypoints to jump to the new code, but if the latter then you need to make sure it's ok to have a mix of threads executing old code while the new code comes online
- data structure compatibility -- if structures have changed you may not be able to hot-reload, but you need to at least be able to detect incompatible changes, and at best you need to plan how the hot-reloaded code will upgrade extant data structures on the fly
- and more
Sure, if you atomically hot-patch all the calls to old code or old code entry points, then threading issues on the side of the hot-reloading library go away, but you still have to think about how old and new code running concurrently will deal with each other.
Now, I don't know that much about this subject, and I've seen how old Lisp and Smalltalk systems could hot-patch/reload all the live long day with no problems, but I suspect all the systems where it was ever easy were a) single-threaded, b) written in high-level languages, c) still had some reloading considerations for the authors of the code being reloaded.
Well, one counter-example: Erlang.
But Erlang is typically the exception to any rule.
Solaris dbx did have this capability -- patch and replace code with newly compiled code from your updated source files -- circa 2000. You just typed "fix" and it worked. I've been wishing for that capability on x86 Linux ever since.
Sadly too expensive for their day, and eventually died, only for us to start having some of those features almost 30 years later.
If you store all your data in mapped files, you can remap them after execve and continue where you started from. This works if the data layout hasn't changed. If you have per type memory maps, you can explicitly convert only the data that has changed. You might need redo logs for things that can't be persisted (networking for example).
Not easy, but also not impossible. All your other concerns still apply of course.
This also has the advantage of not requiring platform specific code and being able to create checkpoints and time travel.
(rant incoming)
This is actually something that bothers me with contemporary programming languages - the huge amount of implicit state due to function calls on the stack and the way the heap is organized. I'm much more in favor of a data-oriented design where you don't get to freely allocate objects, but need to strictly organize them according to a model. I imagine this would also make formal verification a lot easier, since you have a single global coherent view of the program state, kind of like being forced to store all important program state in a database (except more efficient since it's just memory).
Ironically I've become a big fan of global variables. In my experience the classic criticism of "anything could modify anything" doesn't really hold - using globals enables me to statically analyze all usages, not to mention setting memory breakpoints becomes much easier as well - I always know what I want to watch, instead of it being some pointer that will be allocated at some unknown time, 20 stack frames away from the actual usage.
It worked well with fixed size objects, although if I would do it again today I would still allow for a separate pool for variable size strings as updating the max string size was the most common change that both breaks the file compatibility and bloats the size.
The separate pool would make it more awkward to dereference the string as you can't neither use a dumb pointer nor the self offset trick but you have to pass the poll base offset explicitly (or implicitly with a global), but I think it would be worth it.
edit: thinking about it, interleaving fixed size string pools with the other objects might also work, in practice you get a persistent heap.
1) state machine esque predictable function stack
2) global rather than nested state
3) relational model rather than explicit memory objects
4) serialize to disk as builtin
I think
1) is a goal to aim for for the programmer, but when a stack is needed a stack is useful, so should be provided by the language
2) is a good point, encapsulation is a good support for programmers, but encapsulation in the debugger is unwanted; so hopefully the language provides easy support for this
3) Nice idea. Depends on the application domain. Maybe pointers, maybe relational is more useful depending on the situation. Formal verification, err... An excuse for pointers would be: code a solution close to the problem is best for verifying problem->solution->proof which may mean pointers, although verifying solution->proof is better for relational. A good idea to explore...
4) Unequivocal yes
See https://github.com/TomSmeets/quest-for-nothing/blob/master/a...
You have to be careful with global variables however, they will get cleared to zero when the code is reloaded. To solve this either don't use globals or just restore them on reload.
There's no easy way to retrofit hot-reload into these large codebases. Sure, it's easier if you design everything up-front with hot-reloading in mind, but it's still going to be hard to convince developers that they cannot use globals, thread-local storage, function statics, or other inherently stateful things that just won't work with DLL reloading.
EDIT: The deep dive[1] on their webpage is absolutely worth a read.
[1]: https://liveplusplus.tech/downloads/THQ_Nordic_Dev_Summit_20...
Simple dll reloading can work great if you separate state from logic, like shown in handmade hero. If you cannot enforce that then live++ is a good alternative.
Too bad there's no linux support.
Live++ ("Live Coding" in Unreal) can still cause these problems if you leave the Reinstancing option on. For anyone doing C++ in Unreal, please please please read this [1] and at the very least make sure to not use the older Hot Reload.
[1] https://landelare.github.io/2023/01/07/cpp-speedrun.html#com...
It was a C project and it really is not that hard to do (once you get over the thought process and tinkering)
Once I got the hang of it I could whip something together within a few hours.
There was sections for- input, motion, renderer, sound, and frontend/gui. Some allowed to loading multiple things.. others a single instance.
The input one was a good example. I had keyboard using SDL2. One day, someone had a XBox controller in their bag and asked to borrow it while watching a movie. I added support for that. So, when the game loaded, you could specify you are using Keyboard and XBox controller.. so it loads them. If there was a bug, I could modify, recompile and reload.
For the renderer, I had opengl by default... and never got round to doing a Vulkan version. Would have been interesting to have them 'swap' during a live play to compare and tinker.
(the motion was for the mouse. Without really looking into it, I was kinda hoping that would have been a nice setup for VR support but never owned one. Might be something in 2024)
This Live++ looks cool and maybe useful for companies that are spending a bit of cash on decent tools (Unreal, etc) but for small projects, I dont think this is needed.
To be honest, I did not think what I was doing was called 'hot reloading' until, a few years later, I came across Casey Muratori's Handmade hero series. Atleast, I believe it was from there - or other variants at that time.
There are lessons learned, though. I was creating header and src files (with makefiles) for each implementation. If I had to revisit this sort of this, would keep it header only... keeping the build minimal. Might not work.. something I would try, anyway.
But I would have liked to see a comparison of this solution to others on the market, like the readily-available "Hot Reload" in Microsoft Visual Studio.
For a multi-MLOC codebase, VS "Hot Reload" required some patience, and it mostly worked only for simple cases like changing "if" statements or constants. But that's actually all I needed at that time, and considering that I was using VS anyway, I was happy with not having to pay extra.
So, did somebody try out the Live++ hot-reloading technology and could share their experiences?
In our (and many of our customers') experience, E&C doesn't work for anything larger than simple toy projects. It just isn't reliably.
On top of that, I think its biggest shortcoming is that E&C doesn't work with optimizations at all (this is documented), and is tied to the Visual Studio IDE, which not everybody uses.
To my surprise, this was seen as a pretty amazing feature at the time. I never understood why many/most of the other MUDs at the time didn't utilize this.
Not being able to do this on Linux (and it unfortunately looks like this is also Windows only) has been a major source of frustration for me when writing C++, especially given that something like it (edit and continue) has been available in Visual C++ for near on 20 years.
EDIT: Oh, I guess After Effects with some stock background video and audio track mostly.
Burning logo effect: https://www.youtube.com/watch?v=wkP-IYmPzpI
Glitch effect: https://www.youtube.com/watch?v=7eOMSuPdbok
Simulated monitor recording (this doesn't show the monitor border though..) https://www.youtube.com/watch?v=JOYdGtyxMkI
Animated font effects - surely these are possible if the rest is.
At 1:04, I think they made a custom graphic to that overlays the stock video below.
In our (and many of our customers') experience, E&C doesn't work for anything larger than simple toy projects. It just isn't reliable.
On top of that, I think its biggest shortcoming is that E&C doesn't work with optimizations at all (this is documented), and is tied to the Visual Studio IDE, which not everybody uses.