Hacker News Reader

Comment by stocknoob | Hacker News Reader

New things will still be documented, which LLMs can digest. The documentation may have been auto-generated by a different LLM, no problem.

Incorporating recent info into an LLM (fine tuning) is a solved problem.

View on HN

If documentation were adequate, there wouldn't have been a need for Stack Overflow in the first place. At least for now, it still takes (human) intelligence to consume documentation and synthesize knowledge through experience with new technologies, at which point it can be communicated to others. LLMs may get to that point, but they aren't there yet. Also, good documentation explains not only the what and the how of a codebase (which LLMs may be able to summarize from the code), but also the why and the when (e.g. when to use one method or another).

The “why” is another great example too. When you document something you try to do so with some way of showing intent as well. While LLMs are so amazing (really). There is still a lot that I get curious about how it can be effectively used.

Maybe, however there seems to be a limit to that no? I’m not talking about errors that are fixed or factual information. More like discoveries, ex: the FairPlay issue wasn’t documented from Apple it was discovered by multiple users who found and solved the issue I had. How does something like that exist with an LLM? There wasn’t really an error message that told you what to look for. It was a community effort of trial and error. Stackoverflow is nice because of the collaboration that can go on.

Do LLMs "learn" how to code by reading documentation, or by reading code? I was under the impression that it was more the latter; a programming language or library which was documented but had few or no examples of real use in the wild for the LLM to consume would (I think) be hard for current LLMs to work with.

Unfortunately, I can't think of a real-world example (library where ChatGPT has probably seen the documentation, but not any source code) to try out.

This, and I believe this is what Phind does. https://www.phind.com/

Reply on news.ycombinator.com