I moved my home directory to a new computer and Signal Desktop started like if something changed. Sure you would lose your messages on your phone, but you can still access them on your computer if needed.
On a rooted Android phone, you could use oandbackup to backup Signal. If you care about these things, maybe consider using a rooted Android phone?
I agree with you on the centralized platform aspect and the use of phone number (which is both a blessing (this makes it easy for new users to join) and a curse). I also agree with you on Element's UX, but it's getting better and most people can use it fine. I have a few groups on both apps.
I personally prefer Element, which seems more open than Signal and which I can actually use correctly on the PinePhone. And definitely didn't requires the hoops I had to go through because I don't have an Android phone or an iPhone. Axolotl [1] still needs some work.