Violating TOS, at least to scrape and use later, is legal.[0] I'm not sure how the ruling interacts with LLMs, but I'm sure OpenAI's lawyers would bring it up.
[0]: https://www.forbes.com/sites/zacharysmith/2022/04/18/scrapin...
[0]: https://www.forbes.com/sites/zacharysmith/2022/04/18/scrapin...
I see they went to the Supreme Court who kicked it back to the Ninth who then re-affirmed their position that HiQ Labs was not in violation of the CFAA.
[0] https://www.natlawreview.com/article/court-finds-hiq-breache...
[1] https://www.natlawreview.com/article/hiq-and-linkedin-reach-...
So at a federal level, it seems relatively clear. The only uncertainty is on the state level.