But it's "fooling" you into thinking that it's been "fooled". There is no tricking. You just didn't prevent the model from presenting this as an output.

Get real! It's not an attack it's just honestly laziness on the part of you since this must be your chatbot that you have trained and deployed. Consider all of the things that a user could input and if you don't want the program outputting to the user a certain type of output then you have to make sure that output can't be a possibility.

Chatbots are trained bullshitters. Sociopathic bullshitters. They tell you, the user, what it thinks it should tell you.