There's a lot I could say about simplicity here, but here's the short version. There's a few different levels of discussion that we could talk about simplicity. At the lowest level, for example, materialism and idealism are on a par -- they each posit just one fundamental substance from which they think we can explain everything else. As you go up levels of discussion, they are either on a par or it is impossible to determine which is simpler. Simplicity on this particular point is so high up the chain as to not raise any concerns.
At any rate, our theories should be as simple as possible, preferably, while still accounting for everything they need to. Materialism cannot account for the mental, and so it doesn't matter if it's simpler here anyway because it can't do the job.
> A lot of the arguments against materialism just wither away like snow under a blowtorch when you consider the effect of drugs on consciousness.
I agree that drugs can have dramatic, massive effects on our consciousness and experiences/phenomenology. It's just not relevant to the question at hand. None of us disagree that the physical affects the mental, so I don't know how this is could possibly be any kind of argument against arugments against materialism.
> I don't see why this is impossible in principle.
There's two things we can do on our views: show where something is, or explain it away. For the physicalist, they need to show us where the mental is or explain it away. For the idealist, they need to show us where the physical is or explain it away.
The physicalist cannot show us where the phenomenology of the mental is without introducing it into their ontology, and thereby becoming a dualist view. It cannot explain it away either. If there's anything I'm most sure about, it's that there is phenomenology.
The idealist explains away the physical, by pointing out how we merely have the experience of being in a physical world, without there actually being an independent physical world. The idealist can rise to this challenge in a way that the physicalist cannot.