Hacker News Reader

that's because they are distilling the frontier models

Frontier models aren't released, they are closed source.

And the Chinese have been a huge source of innovation in the field.

distilling does not require the models to be released. They simply use the apis.

They have been a source of innovation but probably not in training them.

They just lack of performant hardware. They have enough knowledge. And so they choose a more effective strategy without wasting resources on training from scratch.

You can do a rough distill through the APIs. You don't need the weights.

It was much easier when companies had models on the /completion style APIs, because you could actually get the logits for each generation step, and use that as a dataset to fit your model to.

That isn't to diminish the efforts of the Chinese developers though, they are great.

is there any evidence that this is significant contribution?

My intuition that one need ALOT api credits to distill such large models.

Reply on news.ycombinator.com