I think those who object to the metaphor do so because they interpret "technical debt" as "technical
consumer debt". Consumer debt being a relatively benign form of debt, they don't believe it conveys the gravity of the matter at hand.
Consumer debt is mostly unsecured, almost perfectly fungible across term structures and refinanceable on a whim within a very competitive market of lenders. It's rare for it to be essential for a consumer to take on debt to continue to live[1] and there are typically safeguards in place to ensure consumers don't take on too much debt burden. The only debt that most consumers will come across that imposes any restrictions, or moves slowly, is a mortgage on a residential property. When we talk of "technical debt" we are talking of something has none of the nice properties of consumer debt. If I had sufficient cash I could pay off all my consumer debt within two minutes with a few taps of my phone screen.
On the other hand, I interpret "technical debt" to be akin to a complex structured financial product. Maybe there are convertability clauses between debt and equity, perhaps there is variability in the rates payable, or optionality that can be triggered by various real world events. This form of debt is not liquid, it is not refinanceable and it can be hard to find someone to take it off your hands for any amount of money! Nonetheless, sometimes it is essential for a business to take on this sort of debt to continue to thrive. That is an apt meaning for "technical debt" in my opinion.
[1] rarity depends on how socialised one's healthcare system is!