The X server manages the physical display and provides a virtual display abstraction for clients. X clients connect to the server and make requests.
The X server is called a server because it is a server and X clients are clients.
It's excusable to be confused by the terminology, as in the quote, of course. Unless you claim to understand the whole "network" thing.