Gold was currency for thousands of years, and its value was not assigned by any state.
When talking about money with no use, such as USD or bitcoin, however, they've got nothing to hold them up but the demands of the government that issued them to be paid in them.
The US dollar is just an IOU from the government that gets cancelled if you owe the US government anything other than prison time. Bitcoin is an IOU from no one, and the only way to spend it is to find a sucker who wants bitcoin.
What holds up gold or bitcoin is firstly the trust of the society that adopts it, and secondly the inability for any party to rapidly debase it.
I’m not a historian, so I could be wrong, but my understanding is that currencies came very early, raw bullion wasn’t really used for normal trade, and silver was much more predominant than gold.
This is how you end up with the US government installing a military base over a huge stash of gold, called Fort Knox. They don't do the same for silver and copper.
The value of the constituent metal is dictated by the price (in count) that the mint pays. Nobody pays less because you could go to the mint, and if anybody demands more, you just melt the coin. It was the count that gave value to the metal, not the other way round.
The only evidence for currency from barter comes from the imagination of economists of a particular bent.