However, strictly speaking, the technique should work well for everyone but especially well for some people.
The human brain can store and recall some types of information more readily than other types. Humans have an uncanny ability to recognize voices, recognize whether we have seen a face before, and we automatically absorb all sorts of information about places we travel to.
We struggle to deal with other types of information, like random series of digits, dates, etc.
All memory techniques work under the same principle: take something that you want to remember, and convert it into a different format which you can remember more easily.
Memory palace technique converts input into a spatial representation. That doesn't mean you have to have a good spatial memory! Unless you have a specific kind of brain damage or cognitive impairment then you have an excellent spatial memory just for being human.
If this technique doesn't work well for you, then the point of failure is your association between the hard to rememeber input and the easy to rememeber format. That's great news because it means there is a memory technique that will work great for you if you give it another try until you can figure out how to make effective associations.
It took me around a year of regular practice to get to the point where it does not take me time and effort for each picture. The key for me was to want to memorise something that was so large that, even if using mnemotechnic was hard it was still way easier than to try and root learn it.
Here I thought memory techniques were essentially data structures for the mind. Memory palace always felt to me a bit like a hash table; some of the techniques I read about for remembering sequences of items or numbers were quite literally building linked lists in the head.
Perhaps counter-intuitively most classical musicians are "visual thinkers" and are very good at sight-reading and less good at playing by ear.