The biggest problem with technical debt is that everyone has a different tolerance for it. Some even have an appetite for it. This is a breeding ground for conflict.
The next biggest problem is that the people who take it on are rarely the same people who end up paying it off. The former tend to get showered with kudos whilst the latter experience stress and poor performance reviews.
IMO the worst kind of technical debt is that which is taken on to route around a business process ("Governance"). So it, sort of, contributes to faster delivery but the conditions which drive the choice tend to persist until a regime change so the compromise sticks around forever.
In an ideal world, you should only agree to take on technical debt if you are also presented with a repayment plan. And that leads to my final problem with technical debt: most people tend to assume that that last part will take care of itself.