SimpleX has end-to-end encryption, fancy mobile clients, and does let you host your own server:
It is, at least, intentionally designed that way, for better or worse.
You can get per-conversation sync by joining multiple clients to the conversation.
It's one of those things where you'd probably want to be less secure and always sync to all devices.