> It makes sense with forever stamps, because inflation doesn't hurt their value.

Inflation doesn't hurt their value, but the government can make the cost of postage lower at any time. This would be expensive and they would likely lose a lot of money, but it would still be a lot cheaper than quantitative easing.

> If everyone used stamps instead of dollars, the government wouldn't be able to just print money to fund things.

I realize that this isn't what you meant, but the government can literally just print more stamps whenever they want. In fact, stamps and dollars are both printed by the same company in Canada [0].

[0]: https://en.wikipedia.org/wiki/Canadian_Bank_Note_Company