The X server is fairly portable, and XWayland is built directly from the X.Org sources.

It also has ports to Windows and Mac OS X ...

It's probably not as difficult as you'd think. Embedded X servers have existed for a long time (that is, running an X server as a client of another X server), and running them rootless (without displaying a "desktop" of their own) is also nothing new.

From there it's just a matter of writing the code to allow it to be a wayland client, which isn't that hard, and then doing translation between the wayland event callbacks and X events.

That will at least get you something basic working. There's a bit more to it, of course, and those details are why it's still actively maintained and worked on, but it's not too crazy.

The wayland compositor itself does need to special-case Xwayland, which brings some complexity.

It doesn't really. Xwayland is pretty much an X server that draws to Wayland instead of directly to the display. You can write an X server that draws to anything: a display, a printer, a Windows or macOS surface, an OpenGL texture...

Xwayland is even based on the same Xorg codebase, it's just that the DDX ("device-dependent X") layer used is Wayland-based.

There was one for Windows, Mac and others, why not Wayland?

And wasn't the X11 compatibility layer one of the first things developed for Wayland, since a lot of DEs/apps didn't directly support it yet?

And AFAIK the only part of Xorg where development happens in the recent years.