This is conceptually correct. I'd be curious to hear how well you translate this into practice on something like https://gandalf.lakera.ai/!
Everything I say about GPT-and-friends on Hacker News is a theoretical argument, based on the algorithms described in the papers: I've never really used ChatGPT or the like, and I've been saying the same things since the GPT-2 days.