Granted, but an airgap would make working with some internal support tool a bit cumbersome :)
Bastion servers if properly firewalled might be OK for a short term solution. The concern there is if you allow unfettered ssh (for example) is someone watching for the inevitable brute-forcing that will ensue?