Yep. You can use Signal Protocol over XMPP by using an OMEMO-compatible client [1]. You can pretty "easily" host your own XMPP server using Prosody [2] on a Raspberry Pi.

1. https://en.wikipedia.org/wiki/OMEMO 2. https://prosody.im