I was just thinking about this exact issue. Before abandoning the gold standard, money was basically an IOU for gold. Since abandoning the gold standard, it's clearly not an IOU anymore. It's just the value in itself. Why does it have that value? Only because we trust it.

And honestly, that's really not so different than gold. Why would the gold standard work? What intrinsic value does gold have? Again it only has that value because we trust it. If we stopped caring about gold, maybe if asteroid mining were to quickly result in 10 times the amount of gold that's currently available, gold would quickly lose that trust; same as when a government started printing money with wild abandon. As long as governments are responsible about the money supply, money is just as good as gold. But only as long as they are.

It represents not going to jail for failing to pay taxes.

As a example of how much the value of fiat currency depends on taxation: https://www.irs.gov/taxtopics/tc420

Even if you barter without using any currency, the IRS wants a cut.