The things that constitute potentially infringing copying are not clearly well-defined, and whether or not training an AI is on that list has of course not yet been considered by a court. But you can make cogent arguments either way, and I would not be prepared to bet on either outcome. Keep in mind also that, legally, copying data from disk to RAM is considered potentially infringing, which should give you a sense of the sort of banana-pants setup that copyright can entail.
That said, if training is potentially infringing on copyright, it now seems pretty clear that a fair use defense is going to fail. The recent Warhol decision rather destroys any hope that it might be considered "transformative", while the fact that the AI companies are now licensing content for training use is a concession that the fourth and usually most important factor (market impact) weighs against fair use.