For me, at least, the overall design needs to come from using the code.

I think for refactoring, you're definitely right as if you give them a good high level sketch you can get them to do all of the more tedious implementation.