This is going to be true for any VPN - even if it were purely IP in IP encapsulation, you'd need an additional IP header to get the packet to the VPN endpoint, so your payload is smaller. But fragmentation probably isn't the right answer (especially since any packets marked DF will be dropped). If the MTU for your Wireguard interface is set correctly then anything trying to push a 1500 byte packet via a Wireguard interface should get back an ICMP packet telling it that that won't fit and adjust its packet size downwards appropriately.

I actually hit this recently, where I wasn't able to stream videos from certain sites over Wireguard. I spend a while with tcpdump and figured out that they were using Fastly as a CDN, Fastly was sending 1500 byte packets marked with Don't Fragment, my Wireguard endpoint was returning a message saying that the maximum packet size for the link was 1460 bytes, and Fastly was then… sending another 1500 byte packet marked Don't Fragment. To their credit when I was able to provide their engineering with logs showing this was clearly a Fastly problem, they fixed it fairly promptly