An "open source server"... are you trolling?
XMPP and Matrix services run open source software such as ejabberd
The integrity of your conversation with someone would then depend on both your endpoints, clients, and the respective server.
Just like email, but for chat. There is no single gatekeeper who is allowed to use the network.
> global network consists of servers run by many independent entities
This is not the case for all the popular chat apps including Signal which uses centralized servers which they run themselves. They clearly see little benefit from this distributed independent server model.
And even that doesn't mean the server is open source.
As I explained earlier if you cannot audit the physical server you are connected to, claiming it's open source is useless. FYI that's literally how the term open source was used in this context!
> The integrity of your conversation with someone would then depend on both your endpoints, clients, and the respective server.
Client to client verification simply works and eliminates the need to also "verify" the server which if compromised introduces an even higher risk of contamination in the trust model (too many co-dependent functions are delegated to the server), not to mention collusion in establishing integrity of yet another device that we need to trust.
An IM platform server can be open sourced. Just like any kind of software.
It's just a matter of publishing your code and, preferably making it possible to verify that the service your users are connecting to is build using the same published code.
Typically, the way it goes is that you implement e2ee such that even a fully compromised server cannot read the clients messages, publish the client's source code, and build it yourself or use reproducible builds. That ladt part is where you can criticize Signal. Whether they publish the server code is mostly irrelevant unless you want to run a separate messenger infrastructure.
Or if you S2S federate with the upstream server. Which is a core differentiator of XMPP and Matrix. Signal server(s) notably supported proper federation during their initial growth-phase but famously closed it off ("The ecosystem is moving").
Similar story as Google [Chat/Talk/Hangouts], which did federate over XMPP before they closed that down years ago.