Well, yes, but then you are relying on IP address based auth of some sort. It's just more stuff to keep around, share, and make sure its correct.
If routing was possible with just the public key, after wg has verified the packets, I would assume it'd be easier to maintain (and more flexible).
For example, I could run a "public" wg peer where peers could just join me without me messing with cidrs and whatnot. I wouldn't care what their vpn ip is, as I may not be their only peer and they could be using any number of internal addressess.