Hacker News Reader

Hacker News

Comment by lukan | Hacker News Reader

Parent Full thread

lukan

"it really emphasises how important fine tuning is"

Or rather the quality of the training data?

View on HN

We don't know since no one is releasing their data.

Calling these models open source is like calling a binary open source because you can download it.

Which in this day and age isn't far from where were at.

A big distinction is that you can built on top (fine-tune) thus released models as well as if they released the pre-training data.

You can also build on top of binaries if you use gotos and machine code.

This seems intentionally obtuse. What you say is true, but it is very obvious that this is much more of a pain than if you had the source code. On the other hand, fine tuning is just as easy, regardless of whether you have the original training data.

One could also disassemble an executable and build on top of it. Not for the faint of heart and probably illegal, but possible unless it was deliberately obfuscated. Compared to that, it is impossible with state-of-the-art methods to systematically extract the training data from an LLM model. Fragments yes, but not all of it.

You can do better - generate synthetic data covering all topics. And to make it less prone to hallucination, use RAG or web search for reference material. The Phi-1.5 model was trained on 300B of synthetic tokens generated with chatGPT and it showed a 5x bump in efficiency, punching well above its line.

Synthetic data can be more diverse if you sample carefully with seeded concepts, and it can be more complex than average web text. You can even diff against a garden variety Mistral or LLaMA and only collect knowledge and skills they don't already have. I call this approach "Machine Study", where AI makes its own training data by studying its corpus and learning from other models.

If you don't know the original training data statistical distribution then catastrophic forgetting is guaranteed with any extra training.

I was going to ask for a reference to support this (although please provide one if handy), but the search term “catastrophic forgetting” is a great entry into the literature. Thanks.

Or shell scripts

You can fine tune without the pre training data too.

Mistral models are one example, they never released pre training data and there are many fine tunes.

We are in agreement -- that's exactly what I am saying :)

Their data is the twitter corpus which is public. Or do you want a dump of their database for free too?

Saying "It's just the twitter public corpus." is like saying "Here's the Linux Kernel, makefiles not included."

Or even "here's the Linux Kernel makefiles, no sources included, enjoy".

Twitter tweet data in itself is both highly idiosyncratic and short by design, which alone is not conductive towards training a LLM.

We should just call it open weight models at this point.

How about "weights available" as similar to the "source available" moniker?

weights available or model available, but yes.

FWIW the Grok repo uses the term "open weights".

> We don't know since no one is releasing their data.

Is anyone else just assuming at this point that virtually everyone is using the pirated materials in The Pile like Books3?

I think it's really, really clear that the majority of the data used to train all of these things was used without permission.

The model is open weight, which is less useful than open source, but more useful than fully propriety (akin to the executable binaries you compare to)

Exactly. Training data is key for any trained AI system. And I strongly suspect that every single company active in these modern AI systems is still struggling with how to tune their training data. It's easy to get something out of it, but it's hard to control what you get out of it.

that's a subtle dig at the fact that they have all of Twitter as a training corpus to use, but we don't know how they weight tweets. which, we know they're not gonna be weighted evenly.

I'm sure just like in X's algorithms, @elon tweets are weighted heavily.

The X algorithm is also opensource, so you can verify before commenting..

just because they open sourced it doesn't mean that's actually what they're running on it though

It's not like he needs boosting, he was one of Twitter's top followed accounts long before he bought them. He's pretty good at getting attention.

And yet it’s not enough to curb the desire to tip the scales.

https://arstechnica.com/tech-policy/2023/02/report-musk-had-...

No idea about the current state, but the open sourcing did show, they were favoring elon:

https://mashable.com/article/twitter-releases-algorithm-show...

And personally I never used Twitter much, but I certainly did not follow Elon Musk when I did - yet I had to see lot's of his posts in my feed. Surely just coincidence.

It's not too hard to believe it is a coincidence when the most followed person on a platform shows up in your feed, especially if you follow tech accounts.

Did you not read the article linked in the comment you're replying to?

> they were favoring elon

No, and that's not what the article says either. They were just tracking how well his tweets were doing versus others. They were not favoring Elon.

"They were just tracking how well his tweets were doing versus others. "

Yeah, and adjusting it, so he comes out best. That was Musks demand, as the other article shows, that is linked inside, after a Biden tweet performed better than Musk:

https://mashable.com/article/elon-musk-super-bowl-joe-biden-...

They officially boost people, who pay a little bit. Elon payed a lot.

And the source is clearly not the production source and never where in this shape - otherwise why sue someone, who open sourced it?

"But, the release of this source code also comes days after Twitter forced Github to take down other parts of Twitter's source code that was allegedly posted by a former employee without the company's permission. So, clearly, there's still plenty of Twitter that Musk still doesn't want us to see."

Also, you probably missed that:

"Zoë Schiffer of Platformer reported that Twitter actually removed part of the source code that affected the reach of Musk's and other user's tweets before releasing the algorithm to the public."

Which is consistent with quite some other statements, also from Twitter itself and the fact, that the source has not been updated in 8 months.

See also this HN comment and discussion about it:

https://news.ycombinator.com/item?id=35391854

"But the underlying policies and models are almost entirely missing (there are a couple valuable components in [1]). Without those, we can't evaluate the behavior and possible effects of "the algorithm.""

Sounds a bit far fetched

So changes in power users stats would also result in audience balancing?

Most likely the code was used for analytics and for tracking balance; Elon was a pain in the ass and asked to have custom analytics for his account and devs eventually added him as an audience to be able to get analytics about him easily. A bit dirty but it works.

Most likely the balancing code is somewhere else and it affects only republican / democrats.

X algorithm Github project hasn't been updated in 8 months:

https://github.com/twitter/the-algorithm

So clearly they aren't running it in production.

Also they didn't open source the list of people who are being artificially boosted e.g. Elon.

> I'm sure just like in X's algorithms, @elon tweets are weighted heavily.

Are you sure or is it the literal opposite and you’re just speculating?

Aren't they usually built on most of the same training data?

Or even how much it was trained on this dataset, the amount of FLOPs.

Reply on news.ycombinator.com