Instead of having distinct tokens, you could have modifier vectors which would be added to other tokens. Think in terms of control, shift, meta etc.
There's an interesting exploration of this here: https://www.lesswrong.com/posts/HEzNZ9gvgYwT3aZFS/role-embed....
Curious if you have additional thoughts, and thanks for reading!