Hacker News Reader

Comment by bandrami | Hacker News Reader

> it’s Chinese and backed by god knows who, and no one cares

Existentially weary infosec guy voice I promise you some of us care, and the rest of you will realize why in a year or so

View on HN

I'm genuinely curious about your view on this. Ziphu AI release pretty capable open weights models. As long as we don't feed information that is confidential or become overy reliant on the tech that's on someone else's computer - what are the other riaks that you see in using Chinese models?

My view is that the same people who pulled off xz are putting God knows what in the weights

That doesn't make any sense. Model weights are literally just numbers you multiply by. To imply that it's any way even remotely comparable to the xzutils build process is...

Well, I would be very technically impressed if someone managed to achieve any form of code execution, especially given unknown levels of quantisation post-release whereas xzutils was interesting mostly due to obfuscation.

I was very technically impressed with xz, FWIW.

Note that the troublesome vector isn't code execution by the LLM, it's instructions to produce vulnerable code

Steering a single model towards that seems, again, technically challenging, especially if it needs to be obfuscated enough to pass code review (you are doing that for LLM code right?) and especially if targeted towards specific fields of use (e.g. critical infrastructure). I still don't see how the threat model could make sense here.

> you are doing that for LLM code right?

Existentially weary infosec guy laughter

OK yeah I can see how that would be a problem then lol.

At least at this point most people using them are using them running on western infrastructure. A backdoored llm would be an interesting thing to see. Like every time it realizes it was running in claude code it installed a backdoor or something. For now seems to be theoretical.

Did you ever read the paper on how Ken Thompson backdoored the original C compiler?

Reply on news.ycombinator.com