HNHacker News
TopNewBestAskShowJobs

gwern

35,337 karma · joined April 8, 2009

Site: https://gwern.net/

About me: https://gwern.net/me

What I've written recently: https://gwern.net/changelog

submissionscomments
gwern··on The math that explains why bell curves are everywhere
A little disappointing. All about the history of bell curves, but I don't think it does a very good job explaining why the bell curve appears or the CLT is as it is.
gwern··on Every layer of review makes you 10x slower
After some chatting with GPT-5.4 Pro, I think the 10x claim might simply be because in the >90% load region, you get the famous unbounded delays. And with realistic loss functions like absolute or squared error (consider that in a queue of queues, delays would add) and distributions like Poisson, you'd encounter massively increased losses if you pushed too far towards 100% load. Without extremely detailed accurate statistics of the sort almost no one has, it'd be hard to distinguish 'by eye' 90% from 95%, say, and you'd just see '10x' everywhere you looked.

(And in a pyramid of queues like many layers of reviews, each layer will wind up being about equally loaded, because otherwise you would get a big benefit from adding/removing capacity, so each layer will slowly be optimized towards its breaking point, yielding the 10x everywhere.)

gwern··on Every layer of review makes you 10x slower
The 10x claim reminds me of various scaling laws in anthropology/sociology, especially the much-debated Haire cube-root scaling law (eg https://gwern.net/doc/sociology/1959-haire.pdf https://gwern.net/doc/economics/1983-stephan.pdf ). I wonder if one could give a derivation from queuing theory where you have an argument along the lines of 'in indefinitely many layers of queues of queues, the total delay is minimized at 90% load per queue which yields a 10x scale-free slowdown'?
gwern··on GPT‑5.4 Mini and Nano
I agree. That's why you should write as much as you can now, if you want to get it into the LLMs (https://gwern.net/blog/2024/writing-online). You never know when the window will slam shut and LLM training goes 'hermetic' as they focus on 'civilization in a datacenter' where only extremely vetted whitelisted data gets included in the 'seed' and everything is reconstructed from scratch for the training value & safety.
gwern··on The Brand Age
https://en.wikipedia.org/wiki/The_Spirit_Level_(Wilkinson_an...
gwern··on The Brand Age
Linux isn't a strong brand at all. Even nerds would struggle to tell you what a 'Linux' is (or as I prefer to call it, since that's every non-smartphone Linux I've ever used, 'GNU/Linux'). And I've never even heard of 'Portuguese tarts', and I've heard plenty about Portugal and briefly considered going there last year for tourism.

Pebble went out of business, incidentally.

gwern··on The Brand Age
Some of the typos and poor typography might be accidental simply because that is so common, but the rest of it is surely deliberate.

Remember that _American Psycho_ is repeatedly hinting to you throughout the movie that most, or even all, of what you see is just Bateman's own delusions and hallucinations, which are inferior imitations of reality due to his own impoverished mind and lack of substance. So there's a lot of little tricks all throughout it to subtly destabilize you: https://www.moviemaker.com/american-psycho-anniversary-oral-...

gwern··on Elon Musk pushes out more xAI founders as AI coding effort falters
It's definitely very valuable, but for what AI model? How does any of that lead to AGI, or even just a good coding agent?
gwern··on Celebrating Interesting Flickr Technologies
https://news.ycombinator.com/item?id=47134407

Thanks for writing it up!

gwern··on Printf-Tac-Toe
List of examples: https://gwern.net/turing-complete

It was probably unintentional, yeah, I don't recall any mentions of early printf being overloaded to do stuff, nor is it clear why you would do that since you're using it in a much more convenient Turing-complete language already (C).

gwern··on Malus – Clean Room as a Service
The solution here seems to be to impose some constraint or requirement which means that literal copying is impossible (remember, copyright governs copies, it doesn't govern ideas or algorithms - that would be 'patents', which essentially no open source software has) or where any 'copying' from vaguely remembered pretraining code is on such an abstract indirect level that it is 'transformative' and thus safe.

For example, the Anthropic Rust C compiler could hardly have copied GCC or any of the many C compilers it surely trained on, because then it wouldn't have spat out reasonably idiomatic and natural looking Rust in a differently organized codebase.

Good news for Rust and Lean, I guess, as it seems like everyone these days is looking for an excuse to rewrite everything into those for either speed or safety or both.

gwern··on Why the global elite gave up on spelling and grammar
I would have to consider carefully if I thought I was a high-enough quality candidate that it would be interpreted as a countersignal rather than a signal.

If I, gwern, specifically, were to apply, I might; because I know I am widely read on HN and I've talked with any number of YC partners etc, and they all know I take care in writing, and so me not capitalizing is a deliberate message rather than laziness or incompetence. They may or may not appreciate the message, but they won't infer the usual things, at least.

If I were anyone else and my application just one of thousands in the flood? You'd better believe I'd capitalize and spellcheck my YC application: https://gwern.net/blog/2023/good-writing

gwern··on Why the global elite gave up on spelling and grammar
Also just differing levels of relevance. You don't talk with a businessman or investor or famous people in general because of their writing; if you made a list of relevant skills, 'proper spelling when quickly texting from a phone' surely doesn't crack even the top ten thousand skills. In academia, on the other hand, writing a formal application properly is a core skill.
gwern··on Show HN: How I topped the HuggingFace open LLM leaderboard on two gaming GPUs
My understanding was that Autoresearch was defined as training from scratch (since it's based on the nanogpt speedrun), not using any pretrained models. So it couldn't do anything like upcycling a pretrained model or the Frankenmerge, because it's not given any access to such a thing in the first place. (If it could, the speedrun would be pointless as it would mostly benchmark what is the fastest fileserver you can download a highly compressed pretrained model checkpoint from...) It can increase the number of layers for a new architecture+run, but that's not the same thing.
gwern··on Show HN: How I topped the HuggingFace open LLM leaderboard on two gaming GPUs
Adding, swapping, or duplicating layers has a long history (eg. StyleGAN, upcycling), and it was pointed out at least as far back as He et al 2015 (Resnets) that you could ablate or add more layers because they functioned more as just doing some incremental compute iteratively, and many of them were optional. (Or consider Universal Transformers or heck, just how BPTT works.) So this idea is not far out of distribution, if at all, especially if you're a LLM who knows the literature and past approaches (which most humans would not because they only just got into this area post-ChatGPT).
gwern··on Show HN: How I topped the HuggingFace open LLM leaderboard on two gaming GPUs
> They almost certainly have never seen regular conversations in Base64 in their training set, so its weird that it 'just works'.

People use Base64 to store payloads of many arbitrary things, including web pages or screenshots, both deliberately and erroneously, and so they have almost certainly seen regular conversations in Base64 in their 10tb+ text training sets scraped from billions of web pages and files and mangled emails etc.

gwern··on When AI writes the software, who verifies it?
> I encourage everyone to RTFA and not just respond to the headline.

This is an example of an article which 'buries the lede'†.

It should have started with the announcement of the new zlib autoformalization (!) https://leodemoura.github.io/blog/2026/02/28/when-ai-writes-... to get you excited.

Then it should have talked about the rest - instead of starting with rather graceless and ugly LLM-written generic prose about AI topics that to many readers is already tiresomely familiar and doubtless was tldr for even the readers who aren't repelled automatically by that.

† or in my terms, fails to 'make you care': https://gwern.net/blog/2026/make-me-care

gwern··on We automated everything except knowing what's going on
Insider trading is misappropriating knowledge. Not really misappropriation here.

Maybe you're looking for 'moral hazard'?

gwern··on Microgpt explained interactively
(Pangram commentary: https://x.com/max_spero_/status/2028620466995220568 https://x.com/max_spero_/status/2028689219112034436 )
gwern··on Is Show HN dead? No, but it's drowning
People are making a big deal of the fact that there's multiple prompts or interactions, but so what? That's true of source code too! The final commit doesn't reflect every last keystroke or thing you ever said while working on it; you may have had a long conversation with your coworkers, but then you kept only the summary part that mattered. It is the final, distilled, comprehensive patch.

So the requirement should be that there is a prompt, or set of prompts, plus documentation, which distill out everything learned or done in a single master prompt which then suffices to build a similarly capable codebase. ('Reproducibility' in the sense of replication, getting roughly the same result, not 'reproducibility' in the sense of bit-identical builds. (This is just 'compaction' on steroids, and something I generally do not have problem doing with my own LLM creative writing, no matter how many iterations or conversations it took to get to the final idea.)

If it s not possible to create such a thing for a codebase, even with the vibecoded artifact at hand, then that raises an awful lot of questions about the quality or what went wrong, and the requirement is doing its job of keeping a Superfund site away from the rest of us.

The v0 is just this requirement. If you find that people are lying about it (which they probably will for the same reasons that they lie about Show HNs not being vibecoded or front page slop not being slop eg. https://news.ycombinator.com/item?id=47205208 ), then you can start enforcing it simply by requiring someone else to have run the prompts and found the final result to be reasonably useful and as advertised.

gwern··on Microgpt explained interactively
One of the downsides of using an expert LLM to write for you is that they know all that perfectly well, even if you don't, and aren't too bothered by such a chunk. It's like reading any Wikipedia article on mathematics... This is the kind of thing that people are documenting in the LLM-user literature in creating an illusion of expertise (or 'illusion of transparency'). Because the LLM explains it so fluently, you feel like you understand, even though you don't. Hence new phrases like 'cognitive debt' to try to deal with it.

(This is also why people like cramming or lectures rather than quizzing or spaced repetition, because they produce a certain 'illusion of depth' https://gwern.net/doc/psychology/cognitive-bias/illusion-of-... ).

gwern··on Microgpt explained interactively
Dude, you literally start the article with

> Andrej Karpathy wrote a 200-line Python script that trains and runs a GPT from scratch, with no libraries or dependencies, just pure Python.

Almost immediately afterwards, you have a section titled "Numbers, not letters". Need I go on?

Interestingly, despite all the AI tics, the opening passes Pangram as 100% human... though all the following sections I randomly checked also come back as 100% AI. So the simplest explanation would be that you are operating adversarially and you tweaked the opening to target Pangram (perhaps through a anti-AI-detection service, which now exist and are being used by the cutting edge, as Pangram is known to be relatively easy to beat, similar to how people started search-and-replacing em-dashes when that got a little too well known), which unfortunately means I now expect you to lie to me in your response since you apparently went that far to start building up clout.

(BTW, how did you accidentally pick 4 rare names which were in the dataset? "Thanks, will fix" is not a real response to that observation. Are you also going to remove all of the 'just pure X' and 'Y, not X' constructions from your posts now that I've pointed it out?)

gwern··on Is Show HN dead? No, but it's drowning
The review queue is not going to work, unless you have millions of dollars to spare. No one is interested in providing such extremely laborious skilled labor to be thrown away on one-off tweaks of someone's self-promotion of their AI slop, forgotten within the day, and if they are, the ever-escalating flood will soon burn out their remaining altruism.

My suggestion would be more radical: if it's AI written, the code is increasingly irrelevant. What is relevant is the prompt or spec: https://news.ycombinator.com/item?id=47198562 'Show HN' should ban "code-only" submissions if not majority human-written, and allow only "prompt/doc/test only" 'Show HNs'.

I am interested in a vibecoded project which was a well thought out goal or an interesting methodology or a striking prompt engineering trick (eg. the Anthropic Rust C compiler targeting the Linux kernel); I'm not interested in a obvious idea where I'm supposed to be impressed by a huge pile of rotting-code-debt of unknown quality and likely a lot of bugs, which is going to be written by a LLM in a few months anyway.

> "Show me your flowchart and conceal your tables, and I shall continue to be mystified. Show me your tables, and I won't usually need your flowchart; it'll be obvious." ---Fred Brooks, _The Mythical Man Month_ (1975)

gwern··on Ape Coding [fiction]
Naturally, when hobbyists spend a lot of time in a single block to get results (as they are unable to parallelize or meaningfully coordinate over multiple invocations of themselves, due to lacking key cognitive capabilities such as embeddings), they refer to it as 'going ape'.
gwern··on Cognitive Debt: When Velocity Exceeds Comprehension
Oh yes, this is blatantly AI-written. Whether this is ironic and discredits the claims is up to you.

But the opening at least makes a good point: AI-written code is right now something of an embezzlement or theft or fraud. Like junk food, it presents all the appearance of coming with correlates like "an engineer now understands the problem-space and has built tacit knowledge and a theory of how the domain/code work"... But it doesn't. The human engineer takes all the credit (and promotions), having put in a tenth or less the usual work, but evades the blame or tech debt.

This is why vibecoding is the inevitable endpoint. The current code market is a lemon market, and we see the exact same kind of dynamics of 'buyers' withdrawing from 'purchases' as they see all these shiny code bases which are secretly lemons, and sellers increasingly get frustrated they can't get a 'fair price' (like anyone to look at their 'Show HN' posts).

We must understand that the source code is no more valuable than a compiled binary, and that the responsible human doesn't understand it and has to trust it, in the same way that we now understand source code but don't understand and have to trust the compiled binary. Today, I understand my Haskell code, but I have no idea what is going on in the compiled binary (nor do I claim to); tomorrow, I will understand my prompt/spec, but I will have no idea what is going on in the 'source code' (nor will anyone expect me to or admire me for 'writing all that code' or be surprised if some seriously mistaken assumption is hardwired into the generated code). Just as no one cares about a binary you compiled today, no one will care about your source code. (Unless there is something special about it, of course. 'I compiled Firefox for my Linux' - OK, congrats? 'I compiled Firefox for my toaster' - OK now I'm interested.)

So, no one will care about the codebase for your 'Show HN' project; you have to instead get them to care about your proposal or your testsuite. If it's any good, they can simply pass it to their own LLMs to 'compile'; no need to trust your crummy dubious already-obsolete 'source code'...

gwern··on Dan Simmons, author of Hyperion, has died
That was probably a mistake. The religion is great (and I say that as a staunch materialist atheist) and _Fall of Hyperion_ had a lot more TechnoCore and filling out the background than _Hyperion_.
gwern··on Dan Simmons, author of Hyperion, has died
When you are that spectacularly wrong, it is. If you can't update correctly, you probably shouldn't.
gwern··on I don't know how you get here from “predict the next word”
Specifically, Cochrane wrote:

> On reflection I have started to worry again. In 10 to 20 years nobody will read anything any more, they just will read LLM digests. So, the single most important task of a writer starting right now is to get your efforts wired in to the LLMs. Nothing you write will matter if it is not quickly adopted to the training dataset. As the art of pushing your results to the top of the google search was the 1990s game, getting your ideas into the LLMs is today’s. Refine is no different. It’s so good, everyone will use it. So whether refine and its cousins take a FTPL or new Keynesian view in evaluating papers is now all determining for where the consensus of the profession goes.

For more recent comments, see https://dwarkesh.com/p/gwern-branwen https://gwern.net/llm-writing https://www.lesswrong.com/posts/34J5qzxjyWr3Tu47L/is-buildin... https://gwern.net/blog/2025/ai-cannibalism https://gwern.net/blog/2025/good-ai-samples https://gwern.net/style-guide

The scaling will continue until morale improves. I advise people to skate to where the puck will be, and to ask themselves: "if I knew for a fact that LLMs could do something I am doing in 1-2 years, would I still want to do it? If not, what should I be doing now instead?"

gwern··on Artist who “paints” portraits on glass by hitting it with a hammer
There is an interesting AI point here: the US Copyright Office recently tried to argue that images generated by a model could not be copyrighted, no matter how detailed the prompt nor how curated, because the artist did not envision the exact output and thus it is merely the output of an uncreative mechanical process. Clearly OP does not envision the exact way in which glass will shatter or frost or crack, and has to repeatedly update and revise based on what happens; are his glass portraits mere mechanical outputs and uncopyrightable?
gwern··on Terence Tao, at 8 years old (1984) [pdf]
Oh, if you meant what is the progress indicator itself, see https://gwern.net/design-graveyard#ordinal-word-counts
← PreviousPage 5 of 34Next →