> Due to our concerns about malicious applications of the technology, we are not releasing the trained model.
That was for GPT-2 https://openai.com/index/better-language-models/
The absolute best case is at we end up with similar situation to modern cryptography, which is clearly in favor of defenders. One can imagine a world where a defender can run a codebase review for $X compute and patch all the low-hanging fruit, to the point where anything that remains for an attacker would cost $X*100000 (or some other large multiplier) to discover.
> Due to concerns about large language models being used to generate deceptive, biased, or abusive language at scale, we are only releasing a much smaller version of GPT‑2 along with sampling code .
7 years later, these concerns seem pretty legit.