Normal window glass is Mohs hardness 5.5 or so, as is unhardened steel (your fork and pocketknife); hardened steel (e.g. a metal file) is more like 6.5. That means you can scratch regular steel with glass and vice versa, with some difficulty. (Try it. I just did.) Brass (your key) is around 3 or 4, so it won't scratch glass.
There are lots of things with Mohs hardness above 5.5, including most common metals (except brass, aluminum, copper) and most rocks.
Sand is almost never made up of glass particles. It's often made up of crystalline quartz (Mohs hardness 7) which can scratch regular glass easily. Gorilla Glass has Mohs hardness of about 9, so quartz won't scratch it.