This has been my point for the last 5 or 10 years. That's why I have a "home phone" with banking apps, 2FA and important stuff installed. It has no SIM card and never leaves home. For everything else I have my "street phone".
I had to buy it just because of that. 'cause even my Huawei P-Smart 2021 (currently in my pocket and not among the others on my desk) can't run some of those pesky apps.
Fortunately, my lifestyle places me out of that use case.