Hacker News Reader

Comment by orwin | Hacker News Reader

But it write mostly useless documentation Which take time to read and decipher.

And worse, if you are using it for public documentation, sometimes it hallucinate endpoints (i don't want to say too much here, but it happened recently to a quite used B2B SaaS).

View on HN

Loop it. Use another agent (from a different company helps) to review the code and documentation and call out any inconsistencies.

I run a bunch of jobs weekly to review docs for inconsistencies and write a plan to fix. It still needs humans in the loop if the agents don’t converge after a few turns, but it’s largely automatic (I baby sat it for a few months validating each change).

That might work for hallucinations, that doesn't work for useless verbose. And the main issue is that LLM don't always distinguish useless verbose from necessary one, so even when I ask it to reduce verbose, it remove everything save a few useful comments/docstring, but some of the comments that were removed I deemed useful. Un the end I have to do the work of cutting verbose manually anyway.

The problem with looping is that any hallucination or incorrect assumption in an early loop becomes an amplifying garbage-in-garbage-out problem.

To translate your answer:

- “You’re not spending enough money”

- “You’re not micromanaging enough”

Seriously?

It can generate useful documentation or useless documentation. It doesn't take very long to instruct the LLM to generate the documentation, and then check if it matches your understanding of the project later. Most real documentation is about as wrong as LLM-generated documentation anyway. Documenting code is a language-to-language translation task, that LLMs are designed for.

Reply on news.ycombinator.com