And yet GPT-4 has no problem crafting new words from whole cloth. If you're making new words rare tokens are useful and the model knows this.