This would work for very small companies...but I'm not sure how one can avoid product cycles forever, even without scrum masters and the like. More to the point, how can you make a good product without something approximating product cycles?
The problem with safety is that no one knows how to track it, or what they even mean by it. Even if you ignore tracking, wouldn't one unsafe AGI by one company in the world nullifies all their effort? Or safe AI would somehow need to take over the world, which is super unsafe in itself.