If you're based in Europe then you're quite right. This is a GDPR issue - identifiable PII in the model, and you can force the vendor to remove it.
Specifically, the AI model has both stored the personal data, and arguably "formed opinions" about it.
The first step would be to contact the people who own the model, and ask them under what terms they're storing the personal data. Do they have an exemption? The answer is probably no.
Then, you make a GDPR Erasure Request - delete my data. At that point, given that the company is based in the US with no European branch, then it largely becomes a matter of politics. If your local information regulator feels like it's worth pursuing, they'd contact the company and ask them to sort it out. At that point it's a matter of realpolitik - OpenAI (or whoever have this particular dataset - I'm not sure) probably don't want this to become a talking point - the process would probably result in them deciding to remove your PII from the training set.
There are bunch of other ways it could go of course - the local regulator may decide to sit on it, or the people in charge of the model might decide to ignore EU rules.