Taxes take money out of the economy. Government expenditure puts money into the economy. Its a basic accounting truism. The rate of inflation/deflation is proportional to the difference between the two.
Think of it this way. If one day the government decides to tax 99% of the money in the economy, you have barely any money left circulating for anyone else and you get deflation(less money chasing the same goods). On the other hand, if the government decides to add a billion dollars to everyones bank account, you have too much money in the economy and inflation(more money chasing the same goods. The government can decide to put either of the policies(or anything in between) into effect at any time, it isn't constrained by its own "savings under the mattress" at any point. If there are restrictions on government action in this direction, they are legislative, not fiscal(because, again, the government is the source of all money, it can never be fiscally constrained, no more than the referee in a tennis match can run out of points to award).
RE mattresses: If you owned a money printing machine, you would not be stuffing money under your mattress, because it would just take up space. You would just destroy money you do not need at the moment, and print more as and when you need it.
RE cities going bankrupt: Of course, the city of Stockton doesn't control the money supply, we are only talking of entities who can issue their own currency, i.e. the federal government. This is also why Greece went bankrupt, because it doesn't control the supply of money its debt was denominated in.