It's a shame that we are now back to the times when microsoft made IE4 (or 6?) part of the OS and completely impossible to remove it.
On windows98 and later I would install windows, open IE, disable every single setting to make it completely broken (no JS, no activeX, require latest SSL or bust, etc... pretty much made it impossible to use or be exploited in the background) ... win10 still insists on opening Edge when I click a link from the settings application despite firefox being my main browser.
Now I try (and fail) to do the same with google on android: Reset my phone, disable a bunch of System apps that completely break everything (i have to press OK twice on a "google service stoped working" every time i enable wifi) And I still cannot prevent any app that want to access the web directly or via chrome "web view" and load anything they want and keep cookies no matter my say on the issue. ...not to mention that it makes it a pain to get updates to the bloatware apps I still have to run to make the phone minimaly functional as the app store insists on me having those Advertisement services running to get updates to the bloatware i already bought with the phone.
So, yeah, you can install APK without Google blessing and their captive store, but that is an option for less than 0.1% of users that will want to put up with that.