Hacker News Reader

We actually ran this train of thought with our startup right now. The problem? Not everybody knows what questions to ask. For non-power users it’s usually better to present digestible information and create docs that answer questions they didn’t even know they had.

The thing is, good documentation captures the difference between what you expect and what something is. LLMs, as they exist now, are mass-producers of cliché. You can sense it even in their prose or those inverted puzzle evaluations [0] , they are simply statistical models picking the most likely options without enough depth to subvert the average. When you ask them to describe a piece of code, you'll only get the rephrasing of what the code already says.

- [0] https://github.com/cpldcpu/MisguidedAttention

I mean, LLMs can be regenerated as needed. With every major release, you might regenerate the documentation. Then, you don't need to have everyone taking time to ask the same questions, reducing the amount of times you need to call the LLM. And thereby you can use a more expensive LLM, like o1 to generate the documentation where you wouldn't want to use it normally