I can think of a few ways of combatting this though.

1) Slack in the system, meaning there is time without explicitly defined tasks that have to be done.

Basecamp does something like this where they take 1-2 weeks off scheduled work after a 6-week cycle to let their employees fix things up [0].

2) A good definition-of-done plus a technical culture focused on quality.

My current workplace has a culture of valuing correctness and quality over velocity, it has been a nice change of pace from my previous workplace where velocity was valued over everything else, leading to lower velocity as our code became worse over time.

0: https://3.basecamp-help.com/article/35-the-six-week-cycle