That entire section seemed to be Graeber saying "Economists have been promoting a myth for the last two centuries, but here I graciously point out their error so that they may be corrected and learn the truth." It never seems to occur to him that he might be tilting at a straw man.
I'm quite happy to agree that Smith got it wrong, but the modern examples Graeber cites don't support his claims. If you read them more carefully, what they are saying is "IF we were to live in a barter world then it would be very inefficient, hence money". They don't say that Barter World ever existed; its just a thought experiment.