I don't know why that's funny. This is not a post about a vibe coding session. It's Kenton Varda['s coding session].
later
updated to clarify kentonv didn't write this article
later
updated to clarify kentonv didn't write this article
> this tool is improving itself, learning from every interaction
which seem to indicate a fundamental misunderstanding of how modern LLMs work: the 'improving' happens by humans training/refining existing models offline to create new models, and the 'learning' is just filling the context window with more stuff, not enhancement of the actual model or the model 'learning' - it will forget everything if you drop the context and as the context grows it can 'forget' things it previously 'learned'.
I do this routinely for large initiatives I'm kicking off through Claude Code - it writes a long detailed plan into a file and as we work through the project I have it constantly updating and rewriting that document to add information we have jointly discovered from each bit of the work. That means every time I come back and fire it back up, it's got more information than when it started, which looks a lot more improvement from my perspective.
You're letting Claude do your programming for you, and then sweeping up whatever it does afterwards. Bluntly, you're off-loading your cognition to the machine. If that's fine by you then that's fine enough, it just means that the quality of your work becomes a function of your tooling rather than your capabilities.
How, exactly? Do you think that you're "deciding what gets built and how it's designed" by iterating on the prompts that you feed to the LLM that generates the code?
Or are you saying that you're somehow able to write the "interesting" code, and can instruct the LLM to generate the "boring and obvious" code that needs to be filled-in to make your interesting code work? (This is certainly not what's indicated by your commit history, but, who knows?)
My prompts specify very precisely what should be implemented. I specified the public API and high-level design upfront. I let the AI come up with its own storage schema initially but then I prompted it very specifically through several improvements (e.g. "denormalize this table into this other table to eliminate a lookup"). I designed the end-to-end encryption scheme and told it in detail how to implement it. I pointed out bugs and explained how to fix them. And so on.
All the thinking happened in those prompts. With the details I provided, combined with the OAuth spec, there was really very little room left for any creativity in the code. It was basically connect-the-dots at that point.
> My prompts specify very precisely what should be implemented.
And the precision of your prompt's specifications, has no reliable impact on exactly what code the LLM returns as output.
> With the details I provided, combined with the OAuth spec, there was really very little room left for any creativity in the code. It was basically connect-the-dots at that point.
I truly don't know how you can come to this conclusion, if you have any amount of observed experience with any of the current-gen LLM tools. No amount of prompt engineering gets you a reliable mapping from input query to output code.
> I designed the end-to-end encryption scheme and told it in detail how to implement it. I pointed out bugs and explained how to fix them. And so on.
I guess my response here is that, if you think that this approach to prompt engineering gets you a generated code result that is in any sense equivalent, or even comparable, in terms of quality, to the work that you could produce yourself, as a professional and senior-level software engineer, then, man, we're on different planets. Pointing out bugs and explaining how to fix them in your prompts in no way gets you deterministic, reliable, accurate, high-quality code as output. And actually forget about high-quality, I mean even just bare minimum table-stakes requirements-satisfying stuff.. !
OK. Replace "[expected] deterministic output" with whatever term best fits what this block of text is describing, as that's what I'm talking about. The claim is that a sufficiently-precisely-specified prompt can produce reliably-correct code. Which is just clearly not the case, as of today.
You are just making assertions here with no evidence.
If you prompt the LLM for code, and then you review the code, identify specific problems, and direct the LLM to fix those problems, and repeat, you can, in fact, end up with production-ready code -- in less time than it would take to write by hand.
Proof: My project. I did this. It worked. It's in production.
It seems like you believe this code is not production-ready because it was produced using an LLM which, you believe, cannot produce production-ready code. This is a cyclic argument.
I guess I will concede that this is possible, yes. I've never seen it happen, myself, but it could be the case, at some point, in the future.
> in less time than it would take to write by hand.
This is my point of contention. The process you've described takes ages longer than however much time it would take a competent senior-level engineer to just type the code from first principles. No meaningful project has ever been bottle-necked on how long it takes to type characters into editors.
All of that aside, the claim you're making here is that, speaking as a senior IC, the code that an LLM produces, guided by your prompt inputs, is more or less equivalent to any code that you could produce yourself, even controlling for time spent. Which just doesn't match any of my experiences with any current-gen LLM or agent or workflow or whatever. If your universe is all about glue code, where typing is enemy no. 1, and details don't matter, then fair enough, but please understand that this is not usually the domain of senior-level engineers.
That's been my personal experience over the past 1.5 years. LLMs, prompted and guided by me, write code that I would be proud to produce without them.
I have tried using AI to make changes to the Cloudflare Workers Runtime -- my usual main project, which I started, and know like the back of my hand, and which incidentally handles over a trillion web requests every day -- and in general in that case I haven't found it saved me much time. (Though I've been a bit surprised by the fact that it can find its way around the code at all, it's a pretty complicated C++ codebase.)
It really depends on the use case.
Out of interest: How much did the entire process take and how much would you estimate it to take without the LLM in the loop?
See again here, you're implying that I or my code is disappointing somehow, but with no explanation for how except that it was LLM-assisted. I assert that the code is basically as good as if I'd written it by hand, and if you think I'm just not a competent engineer, like, feel free to Google me.
It's not the typing itself that constrains, it's the detailed but non-essential decision-making. Every line of code requires making several decisions, like naming variables, deciding basic structure, etc. Many of these fine-grained decisions are obvious or don't matter, but it's still mentally taxing, which is why nobody can write code as fast as they can type even when the code is straightforward. LLMs can basically fill in a bunch of those details for you, and reviewing the decisions -- especially the fine-grained ones that don't matter -- is a lot faster than making them.
> How much did the entire process take and how much would you estimate it to take without the LLM in the loop?
I spent about five days mostly focused on prompting the LLM (although I always have many things interrupting me throughout the day, so I wasn't 100% focused). I estimate it would have taken me 2x-5x as long to do by hand, but it's of course hard to say for sure.
I think you're reading a bit too deeply into what I wrote; I explained what I interpreted kiitos' posts as essentially saying. I realize that you've probably had to deal with a lot of people being skeptical to the point of "throwing shade", as it were, so I understand the defensive posture. I am skeptical, but the reason I'm asking questions (alongside the previous bit) is because I'm actually curious about your experiment.
> It's not the typing itself that constrains, it's the detailed but non-essential decision-making. Every line of code requires making several decisions, like naming variables, deciding basic structure, etc. Many of these fine-grained decisions are obvious or don't matter, but it's still mentally taxing, which is why nobody can write code as fast as they can type even when the code is straightforward. LLMs can basically fill in a bunch of those details for you, and reviewing the decisions -- especially the fine-grained ones that don't matter -- is a lot faster than making them.
In your estimation, what is your mental code coverage of the code you ended up with? Do you feel like you have a complete mapping of it, i.e. you could get an external request for change and map it quickly to where it needs to be made and why exactly there?
I know the code structure about as well as if I had written it.
Honestly the code structure is not very complicated. It flows pretty naturally from the interface spec in the readme, and I'd expect anyone who knows OAuth could find their way around pretty easily.
But yes, as part of prompting improvements to the code, I had to fully understand the implementation. My prompts are entirely based on reading the code and deciding what needed to be changed -- not based on any sort of black-box testing of the code (which would be "vibe coding").
Sometimes I give the agent a vague design ("make XYZ configurable") and it implements it the wrong way, so I'll tell it to do it again with more precise instructions ("use a config file instead of a CLI argument"). The best thing is you can tell it after it wrote 500 lines of code and updated all the tests, and its feelings won't be hurt one bit :)
It can be useful as a research tool too, for instance I was porting a library to a new language, and I told the agent to 1) find all the core types and 2) for each type, run a subtask to compare the implementation in each language and write a markdown file that summarizes the differences with some code samples. 20 min later I had a neat collection of reports that I could refer to while designing the API in the new language.
I guarantee you that Kenton Varda's generators generate more code than any other code generators that aren't compilers. ;)