TDD encourages a form of programming where you build from the outside-in, building layer after layer of abstraction, putting off solving the actual problem until you get to the messy gooey centre where it ends up being a kludge. It's also very easy to make flawed assumptions in your original TDD spec which you don't realize until you get to that gooey centre, having wasted N days in the process.
I honestly don't tend to start anything until I have a basic idea of how it's going to work (not a fashionable idea these days), and find that projects end up with a much saner structure when they're built from the inside out.
So my preference is to start TDD once I've got a very basic version of something working, at which point you can start diving in to all the corner cases.