Wayland happened so badly compared to e.g. the Systemd transition or the Pipewire transition because it was developed by Xorg developers who had spent years dealing with its complexity and countless regressions caused by bugfixes breaking obscure behavior that clients depended on. As a result, they swung too far in the other direction. They came out with a minimal protocol where all the heavy lifting was done in each compositor rather than having a single common implementation. Most importantly, the base protocol was completely unsuitable for nontrivial programs. Features such as screen sharing, remote desktop, non-integer scaling, screen tearing, or even placing a window in a certain position were not supported.

Because Wayland's base protocol was unable to replace X11, virtually nobody adopted it when it was initially released in 2008. The subsequent 16 years have been taken up by a grueling process of contributors from various desktop environments proposing protocol extensions to make Wayland a workable solution for their users. This has resulted in endless bikeshedding and requires compromise between different stakeholders who have fundamentally different viewpoints on what the Linux desktop should even be (GNOME vs everyone else). As an example, the process for allowing a client to position its own windows has been ongoing for over two years, and has led to multiple proposed protocol extensions with hundreds of comments on each.

The transition would have been much smoother if Wayland had the same "mechanism over policy" philosophy as X11 and allowed for software to easily be ported to it, but long-term that could have caused the same issues that the Xorg maintainers were facing when they created Wayland.