Ask HN: Has anyone shipped a self-modifying application with LLMs? | Hacker News Reader

Ask HN: Has anyone shipped a self-modifying application with LLMs?

10 pointsex-aws-dude13 comments

In the past the model for software would typically be to ship a fixed application and then if users need some custom functionality, they would need to install a plugin or create their own.

For non technical users they would have to find an existing plugin or maybe it doesn't exist so they are stuck.

But I can't help but think why wouldn't for example an application just have a prompt box that builds an extension dynamically on the fly (e.g. "make me a UI panel with some buttons that does X")

Obviously it would be "use at your own risk" and YMMV

Have any companies or applications actually done something like this?

I think modifying the real source code would get out of hand but perhaps just generating extensions that have access to a sand boxed set of safer APIs.

View on HN

Yes actually, on my GitHub account I shared a self modifying IDE (it can brick itself) [0]

You can connect a model via an open router or Anthropic app key, and file tickets for an llm to “go to town” on editing the IDE itself

The entire IDE code is editable (everything) in the IDE itself

[0] - https://con-dog.github.io/slices-demo/

Not sure what I’m looking at but I feel like I cracked the mainframe

Haha yes. Try the readme [0] (the ide itself is best viewed on desktop or laptop, I didn’t design for mobile)

But basically it’s an attempt at an ide for someone with ADHD, instead of holding a whole feature in your brain you can look at 1 “slice” at a time (fully self contained feature)

It has the ability to connect an openrouter or Anthropic api-key to get an agent to modify the editor itself by using the editor itself. A ticketing system like a kanban board, instead of a chat log

[0] - https://github.com/con-dog/slices-demo/blob/main/README.md

My passion for prototyping took me to setting up an "app prototyping lab", plus a claude code plugin with the procedures for devopsing the lab and authoring the apps.

I do not literally use a prompt box inside that lab app, but it can sit there too, and then it looks like your case: the agent lives inside the app and builds on request. My agent works from "outside" from economic considerations: I want the coding agent on subscription instead of paying per api call for a chat-"resident" inside the box. So the agent-devops operates from my CLI and uses the lab app to prototype apps inside of it. And the lab hosts and runs them.

An app is a python backend with one declared entrypoint (python is the native backend but recently I added support for node sidecars), plus UI components in react/redux/whatever (each UI piece brings its own npm build, the host builds it into static assets and serves them, so for example angular fits similarly). The app exposes "surfaces" - it can declare a handler for data the box routes to it over channeled websocket and SSE and it can declare its own REST and MCP. That is how the clients reach the app, and its own UI components are such clients too. Auth is configured once at the host level.

App authoring/changing process looks like: I issue a token for my agent in the host app, and then the agent can add a whole new app to the host or remove one, or add/remove/change any part of an app, python or react, or its config, and run one reload command against the host. The host reloads the backend, rebuilds the widgets and serves them, the other apps at that time stay unaffected and are up. So if the ask was "make me a UI panel with some buttons that does X", after the reload that panel is simply there, in the running app's ui. Then the agent tests what it built through the browser or the API. I just look at the running result in the lab's frontend app where I can work with ui parts of my apps directly.

The "sandboxed APIs" concept is here too, and the sandbox is the app itself: generated code belongs to its own app and is available only via the APIs of this app through the host. Since the host scales multi-worker and multi-machine, the generated app has to be written in this understanding (like if it was a fast api app). For the agent the host is a simulator box, but everything in it is real.

I've been pondering having some kind of environment that the language model can just modify the application in real time while it is running.

I asked chatgpt and it said hot reloading in a browser, but thats not what i'm talking about. I'm meaning realtime high performance desktop applications that the user can prompt to change the app in realtime while it is running.

The closest thing i saw to this was something called VibeOS, but thats a whole operating system.

I think that you are talking about smalltalk or any other image based programming language.

You'd need source code for the stack, something like OLPC's stack where you have source for everything that's running and then to glue that to an LLM.

Why would you want the model to modify the application in real time? If you could give real world examples.

I'm not a lisper, but it's one of the oft-touted wins of lisp.

It is here for many decades: https://www.youtube.com/watch?v=eEz08IlcNMg

That's Smalltalk

It's not shipped with it, but wiring in AI to my emacs config has given me exactly this. I drop into my emacs config context, ask questions, add packages, modify functionality, and create completely new work processes. LLMs are excellent at writing lisp.

Arguably that's what BI tools like Tableau and Looker do. Or advanced uses of Excel, which an LLM can help with. The user has data and wants to slice it up and see it in a specific way and have some knobs to play with it to figure out answers to questions as a data scientist.

Reply on news.ycombinator.com