Sam Altman said recently that they are already working on making GPT-3.5/GPT-4 finetunable, they are just limited by the availability of compute (partially since none of their SFT infrastructure uses LoRa).

I had previously assumed it was safety concerns, since I don't see what stops someone from finetuning away all guard rails.