I feel it's more about keeping US's massive investment on AI afloat. Added compliance will allow US to further sanction non-US models (i.e. the Chinese ones) as they can just label them non-compliant.
A thought experiment: robot bees can do everything biological bees do --- pollination for the crops and making honey, but they are tireless and cheaper to maintain.
Would beekeepers and farm owners still want to keep biological bees?
It's logical to serve the best version (quant) of the model at the beginning so that users keep testing it. It is also reasonable to think that the developer of the model tried to test various quant levels by gradually degrading the model's capabilities.
> Also, in that case, there would likely be activations indicating that it is favoring a specific version.
> Maybe you can illustrate a realistic scenario in which that would be a problem, otherwise I don't really understand what your point is in this context.
I doubt it. Can you definitively prove that you can reliably detect the kind of threat I described when model weights are released? Can you be sure that your detector won't miss *any* such sleeper attacks? If not, then that's a threat that will be used to justify the ban of models (open or not) that is not sanctioned by the US government. A model being open doesn't make a difference here.
The model is conditioned / pretrained to use a particular version of a library. The model doesn’t know why — it was just taught to use that version. To the model, it is just some insignificant detail in the grand scheme of things. It’s just like how a model would intuitively favour using English without explicit instructions — it’s not trying to sabotage other cultures, it’s just what it does.
They can just favour some specific versions of some library that's been compromised. Unlike introducing bugs / flaws directly in the source code, they can claim plausible deniability, and it's much easier to implement without compromising the general coding capabilities of the models.
Our dev arm (me included) do use Linux VDI. But most day-to-day business are done in Microsoft's ecosystem (Teams, SharePoint, Office, Copilot, etc). In the industry I am in, you would never be able to use any AI tools unless the trusted platforms offer them as part of their products (i.e. Microsoft's Copilot or Amazon's Bedrock) because InfoSec and Legal departments won't risk authorizing any other providers.
Are you more deterministic than an instruction-following black-box? I doubt it. Also, such organization practice mostly becomes a mechanical routine after the initial experimentation phase. Good riddance, I'd say.
I think for this kind of system to work, there has to be SOME kind of public/shared server to do the coordination. If the inviting node is behind a firewall then no amount of information can enable a guest node to connect to it without a node reachable by both.
Yes, that's what I meant. It's not that the Chinese will never be able to come up with models that superior --- it's simply that they will no longer have the incentive to open-source them once their models take the lead.
Training models are expensive. No one can do it for free for very long.
While that might be true, it is unlikely that open source models' capability will ever surpass frontier models --- if you have the best model (by some margin), then people will want to access it, even if it means going through compliance.
Have you seen legitimate corporates use cracked software? If you do, then your competitors will report you; your employees will blackmail you. The risk is too great.
In the not so distant future, all your coding edits (for work anyway) will be through centralized gateways. Think remote desktop environment where pasting from the client is disabled.
The government will just claim that unsanctioned models have the potential to deliberately introduce security vulnerabilities when working on IT projects (e.g. be trained to strongly yet covertly favoring introducing compromised dependencies when you are not looking).
Then laws will be made to forbid organizations who use models other than those from the sanctioned labs to participate in critical projects on national security concerns.
All of a sudden, no business would risk using open source models anymore.
maybe to avoid getting their legitimate email servers banned by other servers since they host (i.e. being exploited) a growing number of spam accounts.
US will ban American companies from using Chinese models and also ban them from dealing with companies who use Chinese models. “Code produced by Chinese models may be deliberately introduce backdoors and vulnerabilities” that kind of thing.