Ok, so in an ML system you can calculate things like cross entropy loss, which penalizes your model for making confident, inaccurate predictions.

It doesn’t care whether your system is made of LLMs, decision trees, or bananas.

However, the problem is that unlike something like a neural net, you can’t exactly use backprop to improve.

On the flip side if the quality of the prediction doesn’t matter, I might as well have Claude spin up something shiny that does the same thing faster, cheaper, and with 10x the magic sparkles.