Hacker News Reader

Comment by gwoolhurme | Hacker News Reader

Maybe I’m slow and stupid, can someone explain to me how new things will be introduced to new forms of LLMs if it all slowly becomes replaced by LLMs? It seems a bit like an ouroboros of human generated content consumption.

Also, SO is flawed that’s without a doubt, but it still even in the age of LLMs been really useful to answer questions that are somewhat tangential not pure code. I was having a problem with FairPlay DRM for live streaming with iOS 16 and SO had something to point me in the right direction. Chatgpt being trained on older data didn’t have the answer. What happens in the future???

View on HN

LLMs can already browse the web with the help of auxiliary systems, consuming current content. AFAIK GPT-4 has a browser plugin for that, as well as several other plugins for retrieving specific types of (current) information.

A well-trained language model only needs to be retrained when the language changes. New facts can be fed to the model by simply telling it about them, which is how those "plugins" work behind the scenes.

I'd like to know this too. Its concievable that most new web content would be output from an LLM so it would just be feeding on itself. That precise outcome will never quite happen as people will transform and validate this output. However, I see sense in a general worry that people "won't bother" if LLMs can do it all easier than a person. Conversely I also see the potential for it to bootstrap for even greater potential human expression, learning and applied thought.

That still doesn’t answer my question on human to human discovery. Say a site like SO disappears because it’s not viable economically to keep it running. Then having something that scrapes the internet doesn’t seem to help does it? Like a quirk in hardware that a human notices, documents, fixes. Something low level.

How is that situation different from how it was before AI? If a human documents something and then the document becomes inaccessible, the information is gone, regardless of whether the "researcher" is an LLM or a human.

The problem I think I’m not conveying correctly is human to human discovery is made easier with communication hubs like SO. With something like an LLM if that reduces the viability for economically running a site like SO. Is there a gap or period where we don’t see that easy human to human communication? Eventually new generated content from a human has to enter the loop until the AI is up to a level where it can replace humans, but if we see less and less human generated content being viable then we have a clear chicken or the egg problem for new training data

Is it specifically human-human contact thats required to generate new insights or document things? I see no major distinction between that and human-LLM contact (aside from awe) or simply an intelligence pusuing a goal by itself for that matter.

Again I could be a massive idiot, but as I see it? For certain things yes 100%. When it comes to low level things, currently an LLM has no way to confirm behavior of something that doesn't give feedback. The FairPlay example I gave requires you to actually play a video with DRM. Another example I can think of are debouncing buttons on a microcontroller based device. As a super basic example but if you are working on an IoT device and require to know when a pin needs to be debounced, that can only sometimes be found under an oscilloscope. However an open source device could be policed with human-to-human interaction. In fact, I have done so making a PR a long long time ago for a crypto wallet that was open sourced. There are all anecdotal examples, I still see a need for at some level a human to enter the loop, however in a disaster scenario if it becomes unviable to run sites like SO then you slowly lose hubs for that knowledge base.

I didn't consider that but its already begun with plugins and auto GPT. LLMs can test their hypothesis. Next, that might be possible via robotics... so scifi.

New things will still be documented, which LLMs can digest. The documentation may have been auto-generated by a different LLM, no problem.

Incorporating recent info into an LLM (fine tuning) is a solved problem.

If documentation were adequate, there wouldn't have been a need for Stack Overflow in the first place. At least for now, it still takes (human) intelligence to consume documentation and synthesize knowledge through experience with new technologies, at which point it can be communicated to others. LLMs may get to that point, but they aren't there yet. Also, good documentation explains not only the what and the how of a codebase (which LLMs may be able to summarize from the code), but also the why and the when (e.g. when to use one method or another).

The “why” is another great example too. When you document something you try to do so with some way of showing intent as well. While LLMs are so amazing (really). There is still a lot that I get curious about how it can be effectively used.

Maybe, however there seems to be a limit to that no? I’m not talking about errors that are fixed or factual information. More like discoveries, ex: the FairPlay issue wasn’t documented from Apple it was discovered by multiple users who found and solved the issue I had. How does something like that exist with an LLM? There wasn’t really an error message that told you what to look for. It was a community effort of trial and error. Stackoverflow is nice because of the collaboration that can go on.

Do LLMs "learn" how to code by reading documentation, or by reading code? I was under the impression that it was more the latter; a programming language or library which was documented but had few or no examples of real use in the wild for the LLM to consume would (I think) be hard for current LLMs to work with.

Unfortunately, I can't think of a real-world example (library where ChatGPT has probably seen the documentation, but not any source code) to try out.

This, and I believe this is what Phind does. https://www.phind.com/

Reply on news.ycombinator.com