I'm familiar with layers; in fact, I have an extensive layer setup in KDE. But then the problem becomes that I can never remember which key under what layer does what action. So, I basically use four keys from my other layers and have given up on all the others I set up.

My poor memory is my unique problem. Layering won't help. Having dedicated keys that I can see, though, helps me remember what those do. As someone else suggested, I can put custom labels on the extra keys.