That certainly does not justify the installer removing X11 from a perfectly working Mac.
I'm actually glad they made this change. In the past, XQuartz was sometimes ahead of the officially distributed version, which could be messy. Now they're the same thing.
For example, I've written some custom device drives for OS X that were installed in the system path (being kexts and all), those were removed. They weren't specifically compatible with OS X Mountain Lion, so it makes sense.