This is fallacious for several reason.
Firstly, no I do not "have to trust the developer anyway" - I can always not install the app. I'm not starting with an app, adding my trust of the developer, and adding my trust of F-Droid - I am starting with F-Droid, and only installing apps which F-Droid trusts, because F-Droid's criteria for accepting apps are far more stringent than Google Play's - more stringent even than I could feasibly audit myself.
Secondly, trust is not binary. Maybe I trust the developer enough to run their published source code, but not quite enough to run their precompiled binary that they might have slipped some tracking library into. Indeed this is very common and F-Droid strips a lot of such libraries. So even if we take it as a given that I am definitely going to install an app, and my only choice is between installing it from Google Play or from F-Droid, F-Droid is still superior because the builds on F-Droid are built from public source, and I am much more sure that F-Droid themselves won't slip malware into the builds because they have a track record of not doing that.
Thirdly, Android's default app store, Google Play, also trusts a third party by default - Google, who insist on running a tracking rootkit on your computer, which is so much more egregiously invasive than trusting any one app that it renders any comparison with F-Droid moot.