Hacker News Reader

Comment by petcat | Hacker News Reader

> Some U.S. firms remain reluctant to use Chinese A.I. models because of concerns over regulation and data privacy. AT&T researches Chinese models but is not using them, Mr. Markus said. Instead, it is working with popular alternatives made by American companies such as the Gemma A.I. model from Google and the Llama A.I. models from Meta.

This makes sense since corporations require legal certainty, and using an open model from an American company (probably) provides them some level of indemnity, and also someone to sue.

View on HN

The current U.S. regime is also replacing some amount of that legal certainty with regime fealty. Picking Chinese options over American ones probably runs a risk of upsetting their leader. I've got to imagine American companies are weighing this factor in their decisions.

That's certainly one factor, yes. But even before getting to that part I think the bigger issues for Big Corp legal teams is mostly around the legal ambiguity of the models themselves. What representations are made about the training data? What jurisdiction governs the license? If somebody alleges that the model infringes their IP, what rights does AT&T have?

Counterparty risk is a lot more straight-forward to evaluate when dealing entirely within the US, with US companies.

To me, the legal concerns seem blown out of proportion. If you use open weight models (of dubious origin) to generate code, you can still verify them with code review and tests and other methods, used to verify human output, right? That is still a great win, maybe not as much as having AI write all the code, but that would be a reasonable point in control vs quantity spectrum for most solid and well made products.

But if you let LLMs talk to people (customers, for example) directly, then yes, you need an LLM provider that you can hold responsible.

Haven't american companies outsourced manufacturing to China for a long time now? I think they have enough experience with dealing with China.

My knowledge is a few years outdated by now, but I remember digging into this and realizing that most of the chinese open-source libs were license-washing software. E.g. PaddleOCR is licensed under Apache 2.0, a very permissive license, however its models were often-times built on/fine-tunes of less permissively licensed foundation models such as Microsoft's LayoutXLM (Creative Commons Attribution Non Commercial Share Alike 4.0). (Which in my laymans understanding is also a kind of viral license in that changes need to be shared back under a similar license?)

The link is annoying enough to find that I can imagine "Mea Culpa" being an effective enough strategy for businesses moving into the ML/AI field, changing their tune after they get caught, but matured their own software to stand on its own feet.

Regime fealty has always been there in US. The current admin is just more corrupt and throughly incompetent at hiding it.

My boss is building out local AI hosting specifically as a hedge against USGov banning access to frontier models for everyone not on the crony list.

Why play ball with a hostile government when you can host your own frontier models?

Exactly. See TikTok trouble as example and quite honestly, try a local open source LLM and ask it to use profanity, paint nudes - the LLM doesn’t answer the question of it is from OpenAI or Google.

The thing is that needs more attention is reverse engineered a LLM which is highly fascinating. I tried it, but it seems I am not there yet to put it mildly. It requires serious effort.

I am just speculating but can LLMs be sleepers? You write software and it seeds traces here and there under certain conditions that pose a serious security risk.

Or a kill switch?

I don’t know. I distrust Chinese LLMs but even more due to training data.

It is after all not a Western model. Different biases and the might be subtle but nevertheless substantial.

In short: no open source LLM may be usable without additional Finetuning for certain valid use cases.

The real value is versioning and autonomy as well as lot more stable answering despite model rot.

Also testing and the supporting systems are easier to maintain.

It is mainly an infrastructure challenge.

Since most folks still somehow review the code, I find it highly improbable, it would become visible quickly. Unless say compiling Unix parts for example. But thats compiler work and not llm.

Jia Tan would disagree.

The overwhelmingly vast majority of open-source code isn't actually looked at or audited. Yes it's there for all to see, but that doesn't mean it's doing any good at the moment, in this context.

Its not the code so much as the model. The model itself can be poison pilled and embedded with ideas that make it unfit for use and there isn't a person out there who can look at the weights and tell you what or where the problem is by looking at the code.

this isn't if-else statements, its a jumble of linear algebra and matrices.

I watched a demonstration where one AI was trained to be obsessed with penguins. they asked for a random set of numbers from it. They fed that set into another AI model to analyze and the new model started to become obsessed with penguins.

I dont think open source is the only one to worry about though. I don't know if we even have a way to guarantee any model is secure, open or closed.

Read the license again.

The license you receive when you download Gemma off of Google's website is not necessarily the same license that AT&T gets when they deploy Gemma as a customer service bot. That's the whole point. AT&T can work directly with Google for a licensing and legal framework that provides certainty.

Reply on news.ycombinator.com