While many here would hate it, Goodharts law is a powerful argument for things being obtus, vague and abstract. The less measurable something is, the less it can be easily hacked.

Also, interestingly, what will stand in the way of AGI - no definable loss function for these vague cases and scenarios.