Measure once, cut twice, earn one to throw away.
In software the material cost is not the cost of a concrete material such as wood (like you mentioned) but developer time
The other thing to bear in mind is that there is always more work to be done than time to do it. You have to factor in opportunity cost. As many other commenters have said, ultimately it's a balance which depends on your company's particular situation.
Thanks to version-control software it's also incredibly cheap to experiment freely with an existing code-base in a completely non-destructive fashion. I see no reason to treat existing code as fragile or precious.