To be honest the limited popularity of F-Droid also helps it be less targetted by bad actors. If it was more popular I would bet the situation would surely be different
Of course the FLOSS aspect adds an additional hurdle that this popular piece of software will have to somehow avoid having much of a contributor community around it since that would greatly increase the risks of your malicious changeset being reviewed. I guess what happened with XZ was about the best case scenario that an attacker could realistically hope for.