This is very true, just want to add the caveat that the notion of low cost of foregone opportunity you are talking about could apply to mature-stage companies, not necessarily fast-growing ones.
Also there is tremendous selection bias in which of these "I turned down X to do Y" stories get told, of course.