As for vocab, for example I would go for a walk in the park and create memorable mnemonics for the vocab words I saw and the park itself would be a memory palace. Later when I wanted to say the word "vending machine" I would pull the Heisig mnemonics for the vending machine I saw in the park and recreate the word. Today when I'm speaking Japanese I don't have to think about about mnemonics unless I've forgotten a word, then I kind of "look it up" in my memory palace.
The thing about the memory palace method is that it's not really fast recall, it's more like keeping a notebook in your brain until it has been adjusted to your working memory.
But for arbitrary random lookups (by some associative key), does it also work?
That's also what note taking software is about, to easily search through the notes, and find related notes. I'm not sure if such interlinks are supported well by memory palaces.
I'm afraid this won't work with Aphantasia https://en.wikipedia.org/wiki/Aphantasia :(
Just as a quick exercise, try to count the number of windows in your house, with your eyes closed. Even if you have Aphantasia, you can probably still do it, even if you don't see the windows themselves in your mind's eye.