I wonder what RMS would say. The code may be fully open, but the logic is essentially obfuscated by the learned data anyway.
https://salsa.debian.org/deeplearning-team/ml-policy https://deepdive.opensource.org/podcast/why-debian-wont-dist...
He would also have something to say about not using AI for critical decisions and allowing folks to appeal AI decisions etc.
> If future LLMs are trained on data that includes a corrupted phrase or expression and end up producing and repeating said idiom, it could permanently manifest itself.
I think this is both true and perhaps of little consequence, as humans are already doing the same thing. One example is the phrase:
> I could care less.
The exploit in the article only has the first of those.
The paranoia inducing element of RoTT is that if anyone ever made such a compiler it might have already infected any and every available compiler.
This has already happened in various areas. For one area, look at gambling (two decades ago). Ron Harris worked at the Nevada Gaming Control board as a tester of new systems and configued the field verification of those systems. Eventually he turned that into video gaming devices that would hit a jackpot after a certain button sequence was pressed.
Yeah, most people bootstrap using a pre-existing compiler, but I know at least one person who compiled their initial compiler to ASM by hand before using it.
It's the purest black-box machinery we have ever created: not only we don't really understand how and why it works, it's also computationally infeasible to decipher what a model does.
Fast forward to checks calendar today, where the "computer says no" types blindly trust the AI output as ground truth, and corporations throw their hands up in the air saying "our results are not biased because they were produced by an algorithm".
Yay.