Transport level:
* Sun RPC [1]. QNX still uses this. It can run over UDP or over raw Ethernet. It just transfers an array of bytes and gets an array of bytes back - marshalling is a higher level problem. It handles messages bigger than one packet, and retransmission. It's simple and performance is good, but there is no security.
* Stream Control Transmission Protocol. Telcos still use this. It's how Signalling System 7 is sent over IP.
Although many implementations exist for both, they've never been popular outside their niches.
Marshalling level:
* CORBA - you define interfaces in a special language and compile them. Data is not self-describing.
* SOAP, from the XML era.
* Google Protocol Buffers - another system where you compile definitions.
* HTTP/JSON - where we are now.
Plus a lot of Microsoft-specific stuff.
Either you have a system where both ends have to exactly agree on format, you have a verbose format with unneeded description data in every message, or you have a very complicated negotiation at connection time. This leads to much disagreement.