There are going to be apps that only run on x11 for a long time and will need to use xwayland, so it's unlikely that linux distributions will stop distributing x11 libraries entirely.
However, development of xorg is completely dead so it is pretty likely that distributions will stop support running xorg sessions in the future, and at some point xorg will likely not run on new graphics cards anymore.
KDE has announced plans to discontinue support for running x11 sessions and only runs on wayland in the upcoming plasma 6 version.
For now all the gui toolkits will presumably continue to support x11, but at some point, it is likely that new applications will only care about supporting wayland, and after that, it is possible that future versions of gui toolkits will stop supporting x11.
Maybe "ripping out working code" is a bad thing, but at a certain point people would have to put in extra effort to ensure that x11 stays "working" and if nobody steps up to put in that effort and x11 support breaks, it won't be ripping out "working" code anymore.