Good point.
Someone (I think Martin Fowler) suggested spending 2-3 days upfront sketching the big architectural points. I generally think that is a good idea.
But I do like OP's analogy of UML being used to make the maps that us programmers will use.
I guess I am looking forward to the day when we don't have to make our maps manually.