I would expect any company with more than a few dozen developers would host their code on prem behind a VPN, let alone a company like OpenAI which has very valuable proprietary knowledge
(Ignoring what "on prem" means these days with so much being hosted and run on computers you don't own/physically control) Then you would be surprised. The threshold is probably multiple hundreds or thousands of developers, way more than "a few dozen". The lure of github purely because it's github is strong, and since Microsoft owns github now it falls under the "no one has been fired for choosing Microsoft" vendor selection logic.
But, even if the hosting of the code is locked down behind a VPN, the nature of it requires copying it somewhere else, perhaps to a laptop that gets lost/stolen. But that is an attack vector and risk that is not addressed by having the revision control only accessible via VPN (although it may provide access auditing capabilities to help mitigate/observe where copies of the code exist).