What about the fact that these models aren't just randomly spitting out and taking credit for random images? This seems the most salient point to me — if I used a paintbrush to create a copyright-violating clone of some notable artwork or IP and tried to pass it off as my own, I'd be breaking the law. We wouldn't try to ban paint and canvas and the human arm because it has the potential to create something that infringes on copyright, we'd enforce the actual act.
If these models make this kind of infringement easy, then they are bad products and their users will run the risk of going to court. The whole thing seems like a non-issue.