Do they? "Fair Use" is an affirmative defense, so the only time we're going to get into that is in a court case, where it'll be tested through legal means.
I would say it's even more nuanced: if LLM training involves merely reading a dataset, but it is not strictly necessary to copy, or even store it verbatim to be useful, then does it even fall under copyright protection at all? A lot of computer-based data processing is already immune to copyright issues; you can place a webpage into a server-based cache or CDN, you can stream it across a network, you can cache it in RAM or local storage, you can make backups of things, and all these processing uses don't fall afoul of copyright.
So I would say that we're going to watch the LLM trainers say that the models aren't storing copies at all, and that seems an even stronger defense than "Fair Use". It is a strange copyright protection indeed that explicitly or implicitly prohibits certain types of machine readings, while allowing many others.