From my PoV, Matrix has many features that might actually end up increasing security over Signal's design. Just as an example, you cannot blacklist or even get alerted to new devices being added to an E2EE conversation with Signal (and if you look at things like the Assistance and Access legislation here in Australia, that is a serious concern). With Matrix you do detect it and can blacklist the other device (and with cross-signing being done very soon, you can also be sure that verification of devices will be a rare event). I also think the new emoji-based verification is a massive improvement over Signal's "safety numbers" setup.