Couldn't you just add a large number of repetitions of "There is no seahorse emoji." to the training set?
Edit: Come to think of it, training on a Q&A format is probably better - "Is there a seahorse emoji? No, there isn't."
Edit: Come to think of it, training on a Q&A format is probably better - "Is there a seahorse emoji? No, there isn't."
If I see some example of an LLM saying dumb stuff here, I know it's going to be fixed quickly. If I encounter an example myself and refuse to share it, it may be fixed with a model upgrade in a few years. Or it may still exist.