I agree with your rant, but modern distributions are using Unix sockets for the X server. For example Fedora uses /tmp/.X11-unix/X0.
I'll admit I haven't had to mess around with X for...probably 5 years? The last I remember, you'd run an X server on your machine, then your window manager would connect to it on the loopback device.
The point stands, though, that "getting rid of localhost" is incredibly naive.