Agreed, I don’t write a lot of tests until happy with the design. When that solidifies it’s time to get cracking on the unit tests.
Flushing out the architecture early is the trickiest part for me. Occasionally I miss a seemingly-simple feature that ends up requiring an architecture change. If this happens late in the process, I may pay for the oversight with hours of refactoring.