Model interpretability is going to be the final frontier of software. You used to need to debug the code. Now you'll need to debug the AI.
They make a shit ton of money because of this. If you're working at a place where the code matters more than the decisions that went into it, you're basically working at a sweatshop for people who are desperate for a win and will throw away you and all your code once the MVP stage is over, and that's the only way this "works".
Generative probabilistic AI is not equivalent to a compiler and never will be until we can do this kind of thing completely deterministically. No matter how much you reduce the error in the "model", it's still more error than the error rate of the logic gates. It's completely futile considering the sheer depth of indirection at play, and that indirection is the whole point of software.
He said something like: it’s crazy that you have the C++ standard but then also how a compiler implements that standard. And then it still matters how that compiler is implemented on a particular architecture! It’s always different and the annoying thing is that this is permutative. This is why we use X86-64 Intel CPUs at <name company> and <name compiler> (I forgot the name). That simplifies it but you still need to know 3 things. He also was frustrated how lock free data structures and memory reordering are 2 completely different concepts in practice but not in C++ but I didn’t fully follow him. I wrote 100 hours in C++ so I was already happy I understood everything else.
But based on his rant I can sort of see why compilers feel not deterministic.