ngl999

This heuristic indicates that at most a handful of them will be of marginal value.

The funniest thing is that the uploaded content is encrypted using a key that the users don't have.

edit: quantity qualifier

LLMs are parrots, when they are trained on English data they behave like English / western speakers which carries the __western__ values.

China would never want a shiny model deployed by some solemn state-own institute to pitch western values like democracy or freedom of speech.

So China is better off starting with Chinese content in the first place, but only to find out it shoots itself in the foot, because censorship __removes__ Chinese content from the Internet, and it __prevents__ truth-reflecting, i.e. quality, content from appearing.

When this is realized to be not sufficient, and they have to use English content, the English content needs to be censored, too.

So compared to other models trained on the uncensored set, the Chinese models are trained on the data sets that is likely smaller. And since high quality content is suppressed, there is likely less high quality training sets. Therefore, their models likely suck.

(Meta knows the importance of high quality training sets so that it started distilling its own employees who are smart.)

> produce GPUs that are competitive

The most recently known advanced process in China is some kind that requires multiple exposures in DUV machines. This already limits the cost to a minimum that's probably not economical.

For GPUs to be competitive, there's more to the hardware. Driver also matters. GPU drivers these days are compilers in disguise. How many high quality compiler projects have we heard of from China?

It's unlikely for both the hardware and the software of a Chinese GPU to deliver what's boasted in its marketing material.

Anyone who tried using Chinese models for serious programming tasks ditch them quickly if they have a choice.

> I don’t think the CCP is restricting or limiting the datasets that models can be trained on.

What I meant was censorship limits the amount of high quality training sets by limiting the amount of all training sets from which high quality sets grow out from, e.g. the entire set of posts on Baidu forums before 2017 is gone forever.

> so we can only hope that Huawei

Chinese companies do not have access to commercial-scale advanced nodes. The impact is at best negligible.

In other words, they look good on surface, but suck whenever anyone put them to any serious use. That's why they are cheap, they have to be cheap because they suck.

No, Alibaba doesn't have the right amount of hardware to serve these models.

The part that benefits Alibaba in these open models is to maintain its public reputation so as to maintain its stock price. The core that drives everything else inside Alibaba is its e-commerce.

Simple as that.