The answer to "Does my system need to handle 1M concurrent users?" could be "Yes", "No", "Not for the foreseeable future" or even "I don't know".
Given that 'debt' is a way to, in effect, allocate (possible) future resources today, it's pretty clear why it's valuable, especially in the face of uncertainty.
That's a given.
You don't need the notion of "Tech Debt" to deal with that.
It is a complete given at any given time, during any given sprint, that all the features in the queue are ASSUMPTIONS.
That's just a tautology.
"Does my system need to handle 1M concurrent users?"
Maybe you answer Yes or maybe you answer No, but you have to answer something to make decisions about the work ahead of you.
The worst possible place is to be in the fuzzy middle saying fuzzy things like Tech Debt and rolling along fuzzily.
Many teams -- especially teams that love this term "Tech Debt" -- do exactly this.
But they are willfully navigating in the dark and "Tech Debt" is just a term that makes one feel psychologically okay with their ignorance.
But it's euphemism, that's all it is. It does not aid in rational navigation.