> If the world is all ideas, then it's just as impossible to explain why different minds have a coherent shared illusion of the physical world, or even any kind of communication between minds at all.
It is far from impossible. There are many idealist models. Here is one: there is a central consciousness (call it God), and God is able to give experiences to other consciousnesses, and those consciousnesses can communicate back to God.
On that model, there is a shared world because God gives experiences to these other consciousnesses as of being in a shared world. The other consciousnesses, in turn, communicate back to God what they will, and that influences future experiences that God gives to those other consciousnesses. That gives a shared illusion of the physical world.
Note that whether you're a naturalist or an idealist, you're going to postulate some things as true without justifying them. For the naturalist, they postulate the existence of physical things without explaining why the physical exists at all. For the idealist, they postulate the existence of mental things without explaining why mental stuff exists at all. If the idealist asks to be granted the above simple postulates, then a coherent shared illusion is possible. I'm willing to grant the physicalist whatever physical postulates they need, and from that I would like to see how they explain (or explain away) consciousness.