It was trained using data they didn't have the license to. So will that hold in court? I hope not.
OpenAI could argue that because some of the training data was written by them, they have a copyright claim to the output. However, this is a very slippery slope for them as the entire existence of OpenAI is predicated their use of training data being fair use.
OpenAI can only control the output of GPT-4 via their terms of service. When you sign up to use ChatGPT or other services you agree to certain conditions.
Attempting to control what users do in their own time with public domain information from your website may be a step too far for a click or browse wrap ToS.