Is a coherent, human-interpretable, goal necessary for recursive improvement?

> how can you be so sure that reaching that goal looks like the straight line you envision?

Because the thing pursuing the goal would be able to improve it's ability to pursue the goal. Doesn't that follow from the premise of RSI?

> we have a growing body of evidence for just how bad we are at aligning incentives with intended outcomes.

We are terrible at it! I don't think that will stop us letting something rip in some unknowable direction (with unintended outcomes). I hope that aligned incentives are a prerequisite, but it seems increasingly likely that they are not.