The part that is counterintuitive to most people when it comes to the "server" terminology is that, with X, your end-user workstation (which may be an incredibly dumb X terminal) is the "display server", which means you remote into a server (in the traditional sense) elsewhere, which then acts as an X client by making requests to your local machine to display windows.

The way most people think about it, "client" is your local machine and "server" is the remote machine that has lots of applications and is potentially multi-user, but X turns that backwards. The big iron is the client and the relatively dumb terminal is the server.

X has the terminology the other way around compared to all other consumer facing software.

This is because of its mainframe style history and technically it does make sense, it's just that everybody else does things the other way around.

For the people who weren't around in the ancient mainframe times who end up messing with Linux for the first time, this is confusing for a while.

I think most of the confusion arises because when you are tunneling X via ssh, the X client/server is the reverse of the shh client/server.

Add to that that the user manages the ssh connection while the X connection is managed for them...

I think the confusion is obvious, given a little empathy for the range of people who use computers.

The server is usually a remote machine, especially back in the time when "client-server" architecture was emerging in mainstream (business) vernacular.