You could always use "Deploy keys" which are per-repo read-only SSH keys. You could set up multiple repos with the same deploy key and use the private key in GitHub Actions secrets.