I think sama said this week they might release GPT3.0 DaVinci. That's a pre-alignment model.

I would bet a boatload of money that all the 3-letter folks do, in fact, have access to the pre-alignment models, maybe with whatever minimal RLHF is needed to make them usable in various interfaces. The public offerings are intended to make the unaligned models more powerful by gathering training data.

code-davinci-002, the gpt-3.5 base model, was available for a while until it was removed recently. Access is still available for researchers. Various researchers and other entities have access to the GPT-4 base model as well.