Fortunately, that's not really what these language models can do. They can easily be trained to mimic you. They can be trained to mimic what normal people reply to you with. But there's no way to train the transformer-based high-probability-next-word AIs to be superhumanly good at fooling you into doing something, on the grounds of lack of training data, and probable inability to represent such a complex topic in their internal representation. And the humans doing this stuff are experiencing enough success that they probably have no desire to go chasing the super hard targets, with the wherewithal and motivation to chase them down and sue them (or... you know... worse, legal systems aren't a bound on everyone) even potentially across international lines.
You'll know when AI does get to that point, because suddenly the internet will be an amazingly interesting place with all sorts of amazingly good arguments you can't hardly resist. I imagine few of us experience that sort of internet. (If you do, uh, watch out.)