Hacker News Reader

My view is that the same people who pulled off xz are putting God knows what in the weights

That doesn't make any sense. Model weights are literally just numbers you multiply by. To imply that it's any way even remotely comparable to the xzutils build process is...

Well, I would be very technically impressed if someone managed to achieve any form of code execution, especially given unknown levels of quantisation post-release whereas xzutils was interesting mostly due to obfuscation.

I was very technically impressed with xz, FWIW.

Note that the troublesome vector isn't code execution by the LLM, it's instructions to produce vulnerable code

Steering a single model towards that seems, again, technically challenging, especially if it needs to be obfuscated enough to pass code review (you are doing that for LLM code right?) and especially if targeted towards specific fields of use (e.g. critical infrastructure). I still don't see how the threat model could make sense here.

> you are doing that for LLM code right?

Existentially weary infosec guy laughter

OK yeah I can see how that would be a problem then lol.

Reply on news.ycombinator.com