It should not be seen as a mechanism primarily used to finance government spending (that's what debt is for). For a country the size of the USA, balanced budgets (tax inflows = government spending outflows) create currency issues, since there's demand for government debt as a store of wealth. What can Apple do if the government decides to stop issuing t-bills? They'd pay a dividend, but then that leave the receiver of the dividend with the same problem. Or maybe they'd purchase an existing t-bond for over market value (effectively increasing interest rates).
The government will issue as many bonds as the market has demand for as a store of wealth, but the government also decides the interest rates it is willing to pay. The idea being, the government has better control over inflation, since government spending drives inflation, but high interest rates curb it (though, I believe this is still an active area of research).