You can have many cloud agents as you wish but you must at least have one where you can remotely connect.
It has saved me hours of troubleshooting and polluting "workflow v1.3.56_final_should_work_2" commits
That’s only true if you’re building simple workflows.
A counter-example would be a workflow that builds and uploads Android APKs. When I last checked last year, there weren't any well-maintained Docker images with the Android SDK pre-installed, and there are no updated, publicly available builds for the runner-images: https://github.com/actions/runner-images/issues/176
The point is that it is very difficult to replicate the environment of a hosted GitHub Actions runner, and having to do so defeats the ease of use the platform provides.
I manually maintain flutter and Android sdk on my server.
I've never been a docker fan, I prefer to completely handle my whole stack.
I have scripts to install the required tools and some actions in my scripts are just echoing what needs to be done manually.
With the years, I've found that infra for fully reproducible builds cost too much to maintain for us.
Even if you can ssh into the remote environment that does not cover things like authentication and authorization, you don't just git a GITHUB_TOKEN with the same permissions.
You don't want anyone to be able to publish public facing app from their version of the code that might not be committed.
Some of us remember an era where deployment was copy-paste from the local /bin folder to the /bin folder on production server.
I am of the opinion you should be able to deploy from your machine, just you do not have the permissions to normally. So that if CI ever goes down and you need to push an emergency fix or something you can break glass if needed.