Let's suppose there is no law on the books that explicitly fleshes out liability if you train a model and it does not do as you expect, we'll need to figure out some nearest neighbor matching.
So you will have to shoehorn this novel issue into some preexisting framework of precedent. Maybe something about negligence liability and a common public use item, like a bridge or something. But a bridge isn't as complicated as an ML model and a bridge failure is way more auditable. Bridges don't hallucinate. Also there are unintended consequences to doing this sort of legal massaging, before you know it, campaign finance = speech.
"Hold them accountable" is so broad that verges on meaninglessness. Part of the whole living in a rapidly evolving tech landscape is living with laws that are always a few steps behind the times. Which bleeping sucks, but until we press for a better large scale solution, it is going to keep sucking.