I’ve chosen the RPi 0W direction for a bathroom music player (Loodio.com) I’m working on because there are so many possibilities in the software and the goal currently is to make it so small a wall socket can support its weight. That way you never have to think about charging it, can do OTA updates and already supports so much software with Linux.
Perhaps the overhead is too high and you can already do a lot with Arduino etc?
So the overhead of Linux is not the issue, even with modern Linux kernels taking up substantially more space. But the overhead of any given distro may be.
You can certainly get lots of things done with a more basic device, but at this point if the unit cost of an RPi 0W works for you, it's certainly convenient to use something like that.
That said, you can do a lot smaller device and still run Linux. E.g. the smallest device I've run Linux on (and it was no record breaker by any means) had 4MB RAM and 4MB flash, and a mid 1990's low end x86 clone.
The Transcend WiFi SD-cards that I linked to elsewhere also is a good demonstration of how small you can get a Linux machine (or could, almost a decade ago)
The future is probably to have the CPU integrated in the speaker.
As a result, most bathrooms in the UK only have the two-prong shaver outlets, if that.
Just added a DAC/Amp + speaker inside.
Actually, you only need to have the input/output in place and plan for the wiring at building construction, because the--literally--biggest problem should be the power converter. Guess that's why you complain about weight, too.