Matrix does have great bridging support so it is maybe the way out. With 2 bridges I've reduced myself to 2 chat apps (Matrix + the occasional SMS) which I am very happy with.
It obviously isn't the final solution, but gives me a consistent interface across networks while I slowly grow my Matrix-native network.