Why do we still need to install stuff? Why can I run webpages just fine without installing anything? Installation shouldn't exist, at least not from the user's point of view.
Installed software is considered to have a distributor, ie. a legally responsible entity that can be punished for any shenanigans.
I believe this is an outdated model and that with better tooling we could do what you envision.
Like installer.exe?
Installation-free software doesn't have to be web software.
While there ARE some tools that require a more invasive "installation" process (e.g., VMWare Fusion), the overwhelming majority of Mac software is installed by just dragging the application bundle into /Applications.
(App bundles are just special directories, more or less, so you're moving more than just the file, but it presents as a single thing.)
To remove an app, you just drag that bundle into the trash.
This usually DOES leave behind things like local user data or preference files, but those are inert text files and don't impact perf or machine behavior in the long run. Users typically waste more space on cat pictures.
The tl;dr is that "installers" and "uninstallers" only exist because Windows needs them. I have seen MANY MANY FOLKS come to the Mac and be VERY confused by this. "But you NEED an installation process! This can't work!" Nope. Windows needs an installation process.
I joked, in the 90s, that despite all the monopolistic chicanery from Redmond, their real lasting awful legacy would be the degree to which they lowered people's expecations about how computers worked, and I stand by that.
(Something else not needed on sane systems: "clean up" software. If you don't litter the file system with files when you add a program, you don't need special utilities to run that shit down and delete it later.)
What you describe works for a specific kind of app, the ones that can be easily sandboxed and don't have shared dependencies.
D/ls from the store do what I describe above. The overwhelming majority of Mac apps going back to the pre-MacOS / OSX days work exactly as I described above.
Homebrew and MacPorts are used for Unix software that requires a Unix-style package management.
Notable exceptions: Microsoft office, Google Drive.
Go to your Applications folder. Right-click any app. Click "Show Package Contents." Take a good look around.
There's also a feature gap that Google will probably never close, because why bother if you can't load the volume of data those power users are working with in the first place.