Hacker News Reader

Comment by 79a6ed87 | Hacker News Reader

I've been paying for GPT-4 since 3 hours after its release. The decrease in quality was noticeable just one week later (on top of the cap changes from 50 messages every 4 hours to 25 messages every 3 hours)

I originally assumed that this was due to the increase in demand. It never went back to being as sharp as it was during those first hours of usage

View on HN

Are you sure it wasn't just that the novelty wore off after a few hours of usage? I never really got into LLMs, but I must say at first it seemed like pretty cool stuff.

OpenAI have repeatedly stated the model hasn't changed so how could this happen otherwise?

I mean, they could be lying. Or only talking about the API version and not the front-facing ChatGPT.

I feel like it would be a fairly large conspiracy by the OpenAI team though? In fact, what motive would they have to make the model dumber, really?

If it gets out and you're right, I think it will cause major trust issues with the product.

They probably do not have a motive to make it dumber, but its a side effect of some other agenda, like higher profits.

I'm sure they aren't trying to make the model dumber, so much as they're trying to cut compute/infra costs so they can be profitable.

> I feel like it would be a fairly large conspiracy by the OpenAI team though?

I'm not sure what you mean by "conspiracy", but this sort of thing isn't unknown. All it takes is the employees being bound by an NDA and a marketing team can say anything it likes (within the bounds of legality, anyway) without fear that they will spill the beans.

I'm not a lawyer but I think saying anything about the model would be in beach of the NDA then? Maybe they went and go special permission to say that the model hasn't been changed since launch?

They want to free up GPU's for other projects. And to do that, they need to make the model dumber.

Not saying it's so in this case (or even in the general case), but "marketing" is sometimes a synonym for lying.

I think this is overlooked. Is it getting worse or are our expectations getting higher as we become more familiar with it? We aren't blown away any more and as our requests get more complex and we become more critical of its output, we mistake that for it getting worse.

This is the part of hype cycle named "Peak of Inflated Expectations", at least when it comes to the potential of the tool. I think we are still in the "Innovation Trigger" phase in terms of applying the technology.

These models have been explicitly nerfed since their first release due to copyright considerations. I've mentioned in two previous cases both for [1] code generation and [2] book summarizing.

From my point of view, it is sad that these sort of socio-political constructs (copyright) are hindering innovation. The funny thing is that in say, 10 years, the "pirate" version of LLMs will be way more powerful and useful than the "corporate" versions. Once the RIAA/MPAA has sued all of them preventing them of using their audio/video; after the editorials have sued them to prevent them from using their books and articles, and after every internet site has sued them to prevent them from using their text. LLM models trained on SciHub, Library Genesis and Torrents will be amazing in comparison.

I sincerely wish that some country would apply to information copyright a similar approach to what India does for medicine patents.

[1] https://news.ycombinator.com/item?id=31852138 [2] https://news.ycombinator.com/item?id=36138930

> From my point of view, it is sad that these sort of socio-political constructs (copyright) are hindering innovation

Why not pay authors of the data the LLM has ingested?

Don't a lot of content creators expect payment per use? Like every time someone streams a song on sportify, the artist gets a few pennies.

So should they be paid a few pennies every time the LLM spits out a response that "used" that training data?

And I am pretty sure it's not even possible to really link the output back to training data anyways.

> Like every time someone streams a song on sportify, the artist gets a few pennies

You're thinking of broadcast radio. Streaming is a fraction of a penny!

> So should they be paid a few pennies every time the LLM spits out a response that "used" that training data?

That doesn't sound unreasonable! I understand that current LLMs have no way to report "this token came from this data" but that doesn't mean it's impossible to build. (Ack: this is a full-on proper dunning-kruger, having not looked into it & having zero knowledge of the field.)

But I think it's probably more reasonable to simply split a % of the service revenue across everyone whose data was used. Or pay an up-fee for ingesting the data in the first place.

Generally, it's crazy to name that some people think it's reasonable for these companies to pay for GPUs and CPUs and electricity to run them but not for the data that's the actual core of their service.

Since they are trained on all kinds of Internet content, might be a little tricky to manage paying >1B people

I mean, figure something out?

Didn't Sam want to create UBI? That's one way!

I'm sure it would be easy to pay publishers who pass on the money? How do you think Amazon does things for kindle?

Will you get a check for your posts on HN? Your tweets? No, right? So all this talk about paying people for their data is rubbish, and what is really at stake here are checks between huge corporations that already stole it from you in the first place.

I'm not quite sure which side your on but yeah, I personally have stopped participating in social media since realizing I'm the product.

HN I agree with you, but I feel like I'm getting enough back from this community to make my time worth it.

> I sincerely wish that some country would apply to information copyright a similar approach to what India does for medicine patents.

Care to take this opportunity to explain India's approach to medicine patents, and why you think it's good?

Personally, I disagree with you. I have absolutely no idea why OpenAI should benefit from copyrighted material without paying authors for the usage. If everything OpenAI did was opensource, I'd probably feel differently about it, but it's totally not, so I really disagree with the view that they should deserve special treatment here.

I remember seeing interviews and reading many comments saying that that the copyright data shouldn't be an issue because once the model is trained, it kind of "forgets" the copyrighted material and we're just left with pure, unfettered intelligence...it's not a fuzzy jpeg of the web etc...

I think the copyright angle is on the money though, this is why Bard isn't as good, just as Adobe Firefly isn't as good as Stable Diffusion. The inputs aren't as good so the outputs aren't either.

Google can't come out in public and accuse OpenAI of blatant copyright infringement for legal reasons, but I bet internally, they know what's up.

It's a reasonable hypothesis. Whenever phenomenons are probabilistic, we're poorly equipped to use our direct experience to assess what they do. We see that all the time with pseudo-medicine for instance. And people became extremely defensive about these questions (vaccines, pseudo-sciences...) - which is why you're downvoted.

totally agree -- Bing GPT4 still works somewhat better in my opinion but has declined in quality alongside chat gpt+ too

What about the GPT4 hosted on poe.com. It has the same output pace as it had in march.

Thought they use Claude instead of gpt

Reply on news.ycombinator.com