Hacker News Reader

Comment by mrkramer | Hacker News Reader

Data is valuable so for example; I don't understand why Reddit and Stack Overflow gave their valuable data to OpenAI and Google for pennies, when they could've made their own chatbots and beat OpenAI and Google at their own game.

View on HN

Apparently (according to AI enthusiasts) all publicly accessible data is free from copyright when used as training data. It doesn't really look like "owning" the data is worth very much money at all.

Also according to eg German law, when done for research purposes.

OpenAI would have stolen it if they didn't license it so it's "free money" for Reddit or Stack Overflow when OpenAI or any other comes along and offers money up front.

It’s already stolen in a way from all the users who just wanted to be able to use a forum and had no serious alternatives

StackOverflow is Creative Commons, so until courts/regulators decide otherwise, anyone can probably claim it's fair game to train on it, same as Wikipedia.

Actually,

The "Actually," is extra, but I can no longer edit it.

Stack Overflow did try this but imo their LLM wasn't that good

So after few iterations they decided to give up and get the quick buck? So shortsighted from them.

Do they have much time left? Seems like once the answers are all scraped and trained, that site wouldn't be able to survive long anyway?

The next version of libfoo released after 2023 will have a new set of options in /etc/foo.conf and at some point a human being who knows that will have to answer a question about it for an LLM to know that.

No, future LLMs will ingest the codebase, any docs, and be able to answer the question anyway. That is, if the LLM didn’t generate the code base itself…

There's a lot of wishful thinking going on here isn't it?

All these models still regularly fail to provide me with correct code to use even the most common open source libraries (stuff like numpy and matplotlib), even though there were certainly trained on that code.

LLMs doesn't learn from reading manuals, LLMs learn by mimicking explanations it has seen made by humans who read the manuals. Until you solve that we will still need humans.

Except, no. Not unless docs and codebases are re-written in question and answer format

Something like 1/3rd of Google searches have never been asked before. Maybe something similar could be said of Stack Overflow questions? There could still be quite a need for human expertise. Tbh, without significant coaxing, LLMs tend to tailor their replies as if they were responding to the 'average' user - there could still be a lot of value for two 'non-average' users engaging in conversation.

I assume they thought that OpenAI and Google would have used that data anyways without a clear way to prove otherwise.

It's not their core business model and so far LLMs aren't very profitable. Investing a ton of money into a whole new business area with heavy competition that may be profitable eventually while not swimming in cash is often how companies die.

Reply on news.ycombinator.com