>Maybe I'm just being very philosophically naïve.
Possibly, more culturally naive than philosophically so.
World War 2 and the Holocaust had a profound impact on Western culture, and Westerners have identified that symbol with the Nazis for almost a century. The impact of Buddhist or Hindu symbology in the West, in comparison and response, has been practically nil.
For Westerners, who may never have been to an Asian country prior to Japan's Olympics, to see a manji and mistake it for a Nazi symbol is perfectly reasonable, if that is the only context in which the symbol exists for them.
The change in icons makes sense, if you (you being some hypothetical Japan) want to communicate specifically to those tourists and encourage their engagement with local businesses and culture. Why use a symbol on maps specifically for foreigners to use which would probably confuse them?
You could just keep it there and say "those ignorant foreigners just need to learn what it actually means" ... but that would show hostility towards to your guests, be bad for business and an impractical way to design a map.
Edit: Apparently they decided not to change the iconography for the maps. It will be interesting to find out what impact if any that appears to have on engagement during the Olympics.