Most of the debt is shorter term and must be continually rolled over. So when first moving from low expectations of inflation to high expectations, there is a temporary one-time gain, but after that the government will need to pay higher interest.

And of course when the expectations return to normal there is a period when the government must pay higher than needed rates.