This seems like two ideas conflating into one. The business 'problems' (1) and the problem you're solving in a particular piece of code (2). These aren't usually directly related. For example, if I wanted to build a YouTube killer, the TDD process isn't relevant to the high level "problems" I want to solve (which, as you say, one might not know of yet).
TDD comes to play (from a developer POV) when I actually start to write some code that, say, transcodes a video or provides an authentication system. In those areas, the problem is obvious and contained and TDD can work well.
That means that odds are that you will never see this code again. You'd be a fool to spend any more time on it than absolutely necessary.
Which does not necessarily mean TDD is "a waste of time". If practicing TDD in a particular situation will result in fewer hours spent developing a feature, the upfront "cost" of those hours is not a waste of time if you spend more hours debugging your way through building a non-tested equivalent.
A developer should have a feel for which way works for them. In my case, I know that the time I spend is ultimately lowered through using some TDD principles (though not all) vs tiresome debugging of untested code. The reward cycle of TDD is commonly overlooked. "Code->Yay!->Code->Yay!" beats "Code->Code->Code->Code->2 hours of debugging->FFFFUU!!" any day.
"True" agile means ditching whatever isn't working. Pronto. There are no sacred cows. Everything is on the table.
Agreed, but that runs counter to making absolute statements like "TDD doesn't fit the mold of startups" and that "it's a waste of time." It can fit and it can save time (just not always, sure, or if you're 'doing it wrong' for your situation).