You're both correct. Building grade sand is scarce and its extraction is environmentally catastrophic.
"Building-grade sand" is sharp sand. It is the cheapest sand you can buy. Roman's built things out of their concrete which was full of random bits of rubble, and some of these structures still stand today.