Ultimately how well your program is designed is completely based upon what features you need to develop in the future, what their parameters are, what their return values are, what those return values will be arguments into, and in what scales you need to run all of this stuff.

These are all things a new programmer has no clue about. All you're really saying is an experienced programmer that has done enough researching and equation balancing (X algorithms feeds Y threads and a processor can support Z cores) ahead of time should be able to design a program that doesn't require too much changes in the future. And presumably, not overdesign in unnecessary micro optimizations.

It assumes perfect knowledge.

The closest to refactoring in programming is the ability to use IDEs to remove dead code and do code generation. But programers learn about IDEs long after they would first be useful - that would be around when you get to integrations, which easily triples the code base of your program.

Everything you mentioned is also true about programming