> All in all, a prompt history would be really edicational if anyone thinks about doing something similar.
As we start to develop AI first programs, I believe we will need to start connecting LLM conversations to code, not only for educational purpose but for maintenance as well. I'm currently experimenting with what I call Block-UUIDs, and they are designed to make it easy to trace LLM generated code. You can see what I mean in the link below, which contains a simple hello world example.
https://app.gitsense.com/?chat=7d09a63f-d684-4e2c-97b2-71aa1...
Something worth noting is, you can't expect the LLM to properly generate a UUID. If you ask the LLM, it'll says it can, but I don't trust it to do it correctly all the time. Since I can't trust the LLM, I instruct the LLM to use a template string, which I can replace on the server side. I've also found LLMs will not always follow instructions and will generate UUID and how I handle this is, when the LLM stops streaming, I will validate and fix if needed, any invalid UUIDs.
How I see things playing out in the future is, we will alway link to LLM conversations which will give use the Block-UUIDs generated and by looking at the code, we can see what Block-UUID was used and how it came about.
Full Disclosure: This is my tool