VPNs are primarily used for remote access, to get random machines access to closed IP networks. Service meshes synthesize a new network (sometimes IP, sometimes something else) to connect a bunch of related machines, almost always with policy controls for who can talk to what, usually cryptographic.
It would be weird (but not "wrong") to use a service mesh to get developer laptops access to staging Postgres.
It would be weird (but not "wrong") to use WireGuard to connect an application server to its Postgres instance.
WireGuard is a much tighter and more limited design, intended for integration directly into operating system kernels, with a strong emphasis on performance. Nebula is a much more ambitious design; it includes direct DNS support, certificates, and server infrastructure. WireGuard is a few thousand lines of very carefully written C code; Nebula is a typical Go project.
They're both very cool.