The interesting thing is that a monetarily sovereign government actually sets its own interest rate, via monetary policy: the interest paid on US treasuries is ultimately a function of the interest rate set by the central bank, i.e. it is a political choice.
This is initially unintuitive, but it follows from developing an understanding of the system from first principles, as is done by Modern Monetary Theory. You can read about it under the heading "What role does the bond market play in all of this?" in this article I already linked to elsewhere: http://pragcap.com/resources/understanding-modern-monetary-s... and also in Randall Wray's book "Understanding Modern Money".
Quite a number of economists in the MMT line of thought actually argue in favour of letting the interest rate drop to zero, and using fiscal policy instead of monetary policy for inflation targeting.
In other words, government debt does not need to be a way of taking from the poor. It is that way as a result of political choice - though I do not think that it was a conscious choice, since people have not really internalised how our monetary system works these days.