> borrowing implies issuing bonds, which will lower interest rates
You have it exactly backwards.
Issuing bonds means selling bonds. Selling bonds or anything else puts downward pressure on the price.
When price of a bond falls, its yield (interest rate) rises.