What is a dead program? The talk never says.
In the talk he discusses what are called "live programming" environments, as exemplified by Smalltalk and some implementations of Lisp. Instead of restricting programmers to writing code and then compiling/running it, live programming environments let you modify the program after it has started running.
If you want to make a change, you just modify the relevant bits of code and the program will immediately start using them. There is no need for a build step and no need to restart the program (which would involve losing any state that's in memory).
One programmer fixing a bug, one adding a new feature, are both working on the same program at the same time? Or is each one working on their own copy? And how does the merge happen if they're working on separate copies? How is the updated code moved to production? How is this "live coding" supposed to work?
* There's still a notion of production. Most people work on dev environments just like today. You do need the ability to merge code in some reliable form from one environment to another. The Clerk project mentioned in the talk does this.
* Dev environments are more fluid. You can debug issues faster because you spend less time getting to arcane parts of your program after every restart.
* It is possible to live-edit production for important incidents. It's very much a weapon of last resort, you have to be super careful, and you probably want to rehearse in staging the way NASA does with their rovers. But it has the promise to reduce the amount of time your customers are impacted in major incidents.
This talk inspired me to go build a live programming environment in my current stack of choice: https://spectra.video/w/wkDB5fsjBNBbsqKXGhGzwT
Yeah, that works about as well as saying you don't need tests because it's not very smart to write bugs. You need processes to enforce these things at scale.
Smalltalk, Lisp, Scheme, Erlang, Haskell, OCaml, Java, .NET, C++ (via VS, Live++, Unreal) allow for this kind of interactivity, where Smalltalk and Lisp variants win in tooling.
Some Lisp variants, Erlang, Java (via JMX beans / JVMTI) also allow to connect to a running application in production and change its behaviour, although on Java's case it is only a subset of possible changes.
Listen to Ron Garret’s interview on Corecursive.[1] He sent verified Lisp code to Mars at NASA. The code failed but the debugger popped up and they were able to recover from it. Look for “Debugging Code in Space” and “Sending S-Expressions” in the time stamps.
Ron Garret (https://news.ycombinator.com/user?id=lisper) is a frequent contributor on HN.
There just isn't much forcing people to engage in good software development and operational strategies. Sometimes there are regulations in the field (PCI, HIPAA), sometimes adversaries force care (state sponsored attacks on Google), but that's kind of rare. The usual forcing function is a competitor that delivers features faster than you. In that case, maybe "live programming" is worth a look. I've never seen "less bugs" on a product comparison sheet, for example.
(It's a contentious view, and I'm personally not a fan. I like it for software I'm sitting in front of, like Emacs, but hate it for things that need to run unattended on their own, which is like everything except text editors. And "less bugs" is a big selling point for me personally. Every bug I run into consumes time that I'd rather spend elsewhere. But, not everyone sees it that way.)
Those show up, your sales department just uses the more formal names for the actual customer demands that “less bugs” encodes.
That will look like some combination of
• “We have an SLA for an annual 99.9xx% uptime and average sub x00 latency!” (so if a bug causes significant annual reduction in service, your business gets penalized),
• ”We guarantee regulatory compliance!” (so if a bug or business use causes regulatory issues, it’s your ass on the line),
• “We guarantee 24/7 same day chat or email support!” (so if a bug causes an outage, they have a warm body they can yell at, demand an explanation for their customers, and ask questions)
• “We guarantee backups, data redundancy, and worldwide data replication!” (so if a bug blows up one of your data centers somewhere, or the intern fat-fingers an accidental deletion of the production user database, the customer doesn’t even notice something went wrong)
• “We guarantee API backwards compatibility or a service maintenance guarantee until 20xx!” (So you’re forced to fix bugs, and the bug isn’t ‘your PM team might try to #KilledByGoogle’ the service you invested in building infrastructure on.)
Teams that believe in the health, care, and security of their design are far better equipped to offer the above valuable terms to customers to gain competitive business advantage, and teams with bad engineering hygiene are likely to be scared off.
Neither of these (inherently) have the kind of "live programming" environment that the original comment was talking about.
Jupyter notebooks have a different reproducibility problem though, with how easy it is to create hidden state. But they're not intended to be used in production at all anyway, so the problem mainly affects pedagogical and information sharing use cases.
Julia with the Revise.jl package installed and loaded comes kinda close to the live programming model - but the state is always preserved in the source file necessarily with Revise (and it's a development tool too, not one loaded in production). Since you mentioned it alongside Jupyter, maybe you meant the Pluto notebook for Julia? There too the state is always preserved, both in terms of packages (with Julia's usual Manifest file) and in terms of your code (no hidden state like Jupyter, since it's a reactive notebook).
(random class selected)
It's not that human unfriendly, if you know Smalltalk (which is not a terribly hard thing to learn). But you also wouldn't interact with it in that form but rather inside of Pharo and using its class browser.
Pretty much by definition, more code touches non production environments than is ever ran in a production one: everything that touches production likely passed through at least 1 other environment beforehand. Then you have the code that never made it to production due to it being buggy/etc.
Why? Reliability, auditability, and testing isn't affected. This is about developing methodology and how the language syntax and runtimes can enable that.
A good implementation of the idea would incorporate version control – when a service starts, it knows which version of the code it is running (commit hash, etc), and it has an introspection interface to report that version. If any "live coding" changes are made, it knows exactly which classes/methods/functions/etc have been changed compared to that initial version, and reports that through that interface too. You can then have a centralised configuration management system, which polls all services to find out which version they are running (and whether there are any additional changes beyond that version), and alerts if any production system is running different code from what it is supposed to be. Since your "live coding" interface is audited, you know the exact timestamp/username/etc of the change, and so anyone making inappropriate changes to production systems can be caught and dealt with.
In production, a "live coding" interface can be used to enable live patching – so you can apply a patch to a running service, without even having to restart it. Of course, the patch would be tested first in lower environments, and the interface would be invoked by some patching tool / deployment pipeline, not an individual developer.
I worked with an Object Oriented database that had some cool features because the code was stored as highly structured data within the database itself, i.e. the code you saved was not text at all (except when dumped/reloaded to external file - similar to exporting/importing data of a database to/from a file as a textual representation).
Even back on the DDJ days there were articles about them.
The way I see it, there's two scenarios where this approach can be useful, which I'll address separately.
The first is for local development. When iterating on a solution to a problem, it can be a hassle to restart the program from the beginning and incur the time penalty to get to the part you're modifying in order to test it. This can include not just compile times, but also program startup, loading in necessary data, and going through a series of steps either in a UI or via API calls to get the system into a state where it's about to trigger the functionality you're working on and want to test.
For example, I'm currently working on a business workflow that has a bunch of steps that require someone to fill in several forms, and things happen after each of those. When I'm working on stage 5 of that process, testing it involves repeating all the actions necessary to go through steps 1, 2, 3, and 4. Yes, it's possible with extra effort to automate this, or to create mock state that can be used to jump directly to step 5 on launch, but that's just a manual substitute for what a live programming environment gives you automatically. A similar situation occurs in game development; you don't want to play through half the game every time you want to get to a particular part where you've changed how a particular enemy acts.
The second scenario is production. You're correct that it's important to only deploy code that's undergone proper testing and review, has a known version, and is reproducible. However this is orthogonal to the use of a live programming environment. With the latter, you wouldn't just have developers interacting with it in an ad-hoc fashion (just as you wouldn't have developers directly modifying python scripts on a production server). Instead, the ability to modify running programs can be used as a deployment mechanism. Once a set of changes has been made in a local development environment, undergone testing, and is deemed ready for deployment, the same mechanisms for updating running code that are used for development purposes can be used as a way of deploying code. And in production, the fact that you don't need to restart the process means you can avoid downtime.
As far as deployment and versioning is concerned, you can think of it as being a similar to how you would do those things for a server based on CGI scripts or similar (e.g. PHP) where every time a request is served, the file is read from disk and executed. The difference there is that all state has to live in a database, so if you have long-running processes e.g. business workflows that span days or weeks, all state transitions/control flow has to be managed manually, rather than using language features like sequencing, iteration, and function calls. With a live programming environment that supports persistence (meaning that execution state is stored in a file or database in a manner transparent to the programmer), deployment consists of adding/updating a set of objects in the data store, rather than copying a set of files to a particular location.
An example of a system that supports runtime code updates is Erlang, though it is not persistent. An example of a system that is persistent is Ethereum, though that doesn't support runtime code updates. Smalltalk and its variants support both.
Auditability can be supported by ensuring all changes to code or runtime state are made through transactions.
No thanks.
And it was a complete security nightmare. It had to shut down after about two weeks, due to rampant abuse. One person managed to escape the game world, then the VM which contained it, then wreak havoc on the host machine running several other instances of the game (for linked worlds.)
If you aren't focusing on security at every level, you're asking to get wrecked. You think things are secure, man can make it, man can and will break it, eventually.
I understood you. This example doesn't illustrate the same concept. You're talking about allowing RPC into a running program. The talk is about RPC to your IDE as you develop the program. These are very different situations.
The colloquialism "thinking about security at every level" doesn't mean that shipping a program on air-gapped faraday-caged hardware is the only security. Any machine running a website can have the program modified by changing the HTML (or HTML generating code) at any time. Ruby, PHP, Perl, or even Tomcat (which will reload artifacts in realtime, without some tweaks) are hobbled versions of the same concept. Elixir/Erlang and LISP coding (et al) is live coding due to the nature of the runtime.
This idea of having an interactive program, as you develop, does not preclude a hardened artifact (which has never been the problem, since swapping out a replacement that's hardened would make that pointless) but that's partly the point. Making new toys and features, the talk is about the important elements to keep focusing on and why to move development forward.
You still need proper access control.
At about 22 minutes:
> I want to talk about interactive programming, which is I think the only kind of programming we should really be doing. Some people call it live coding, mainly in the art community, and this is when you code with what Dan Ingalls refers to as liveness. It is the opposite of batch processing. Instead, there is a programming environment, and the environment and the program are combined during development. So what does this do for us? Well, there's no compile and run cycle. You're compiling inside your running program, so you no longer have that feedback loop. It doesn't start with a blank slate and run to termination. Instead, all of your program state is still there while you're working on it. This means that you can debug. You can add things to it. You can find out what's going on, all while your program is running.
And a few minutes later:
> So, for example, in a dead coding language, I will have to run a separate debugger, load in the program, and run it, set a break point, and get it here. Now, if I've had a fault in production, this is not actually so helpful to me. Maybe I have a core dump, and the core dump has some information that I could use, but it doesn't show me the state of things while it's running.