Comment by gwern | Hacker News Reader

Comment by gwern | Hacker News Reader

Parent Full thread

The review queue is not going to work, unless you have millions of dollars to spare. No one is interested in providing such extremely laborious skilled labor to be thrown away on one-off tweaks of someone's self-promotion of their AI slop, forgotten within the day, and if they are, the ever-escalating flood will soon burn out their remaining altruism.

My suggestion would be more radical: if it's AI written, the code is increasingly irrelevant. What is relevant is the prompt or spec: https://news.ycombinator.com/item?id=47198562 'Show HN' should ban "code-only" submissions if not majority human-written, and allow only "prompt/doc/test only" 'Show HNs'.

I am interested in a vibecoded project which was a well thought out goal or an interesting methodology or a striking prompt engineering trick (eg. the Anthropic Rust C compiler targeting the Linux kernel); I'm not interested in a obvious idea where I'm supposed to be impressed by a huge pile of rotting-code-debt of unknown quality and likely a lot of bugs, which is going to be written by a LLM in a few months anyway.

> "Show me your flowchart and conceal your tables, and I shall continue to be mystified. Show me your tables, and I won't usually need your flowchart; it'll be obvious." ---Fred Brooks, _The Mythical Man Month_ (1975)

That's very much in keeping with the proposal I floated last week (https://news.ycombinator.com/item?id=47096202) and again yesterday (https://news.ycombinator.com/item?id=47213630), and I'd love it if we could make it work. Basically what we want is the source code, and generated code by definition is not source.

The problem is that it's not clear exactly what format that takes nowadays. If it were a single one-shot prompt, you could share that. Ditto if it were a single session with Claude Code or whatever. But the iteration loops going on here are much more complex and noisy than that.

So where I'm currently at is: we'd be willing to build something to support this, but what would we build? What would be the a v0 of this or, to use pg's old phrase, the first quantum of utility?

People are making a big deal of the fact that there's multiple prompts or interactions, but so what? That's true of source code too! The final commit doesn't reflect every last keystroke or thing you ever said while working on it; you may have had a long conversation with your coworkers, but then you kept only the summary part that mattered. It is the final, distilled, comprehensive patch.

So the requirement should be that there is a prompt, or set of prompts, plus documentation, which distill out everything learned or done in a single master prompt which then suffices to build a similarly capable codebase. ('Reproducibility' in the sense of replication, getting roughly the same result, not 'reproducibility' in the sense of bit-identical builds. (This is just 'compaction' on steroids, and something I generally do not have problem doing with my own LLM creative writing, no matter how many iterations or conversations it took to get to the final idea.)

If it s not possible to create such a thing for a codebase, even with the vibecoded artifact at hand, then that raises an awful lot of questions about the quality or what went wrong, and the requirement is doing its job of keeping a Superfund site away from the rest of us.

The v0 is just this requirement. If you find that people are lying about it (which they probably will for the same reasons that they lie about Show HNs not being vibecoded or front page slop not being slop eg. https://news.ycombinator.com/item?id=47205208 ), then you can start enforcing it simply by requiring someone else to have run the prompts and found the final result to be reasonably useful and as advertised.

Reply on news.ycombinator.com