I'm not sure that's accurate. Most companies use "AI safety" as a shorthand for "brand safety", i.e., making sure that your LLM can't be trolled into praising Hitler or denying climate change. Basically, do what needs to be done to avoid bad press.

There is definitely that second and more abstract take - AI doomsday safety - that Altman liked to talk about to get favorable regulation passed. But it doesn't seem to be much of a real field. "Hard" AI doomers think that superhuman AI can't be reliably constrained and we should just stop. "Soft" doomers think we need to solve "alignment". But we don't really know what's going on inside an LLM, so we test for alignment by asking it what it thinks about Hitler. In other words, back to brand safety.