Something mentioned in the book "The Toaster Project" (where a guy tries to build an electric toaster from scratch) is that high-grade ore basically doesn't exist any more, and you now need to process a ton of ore to extract a few kilograms of metal.
(Just something to keep in mind if you ever find yourself in a position of having to rebuild civilization after being trapped in stone for 5,000 years.)
I'm not sure what would be considered historically "high grade" but modern high grade ore is 62 to 65% Fe see for example here (first google result for Iron Ore Benchmark price https://www.marketindex.com.au/iron-ore).
There are very large reserves of this stuff in places like Brazil and Australia it is not exactly a finite resource.
Lower quality ore is also traded. There is a 58% Fe benchmark price as well.
You buy ore in one of two formats "lump" or "fines" (https://www.metalbulletin.com/Article/3811904/Iron-ore-prici...)
Lump is considered to be the premium product and is generally +8mm sized product - think of it like pebbles. You can charge lump directly into a modern blast furnace.
The other product fines is more like dirt than pebbles and needs to be agglomerated either via pelletization (https://en.wikipedia.org/wiki/Pelletizing) or sintering (https://en.wikipedia.org/wiki/Sintering)
A well run modern blast furance typically operates in region between 1,300 to 1,600 kg of ore per Tonne of iron produced. This is far and away from "ton of ore to extract a few kilograms of metal".
I think maybe confusion is that there is high demand for ore. Iron Ore is considered by countries like China etc as a key strategic resource and keep their own internal stockpiles. Supply ships (bulk carriers) are booked months ahead of time). So while average citizen can't go into a store and buy a handful of ore to make his own toaster that's very different from saying the stuff doesn't exist.
Depending on the apocalypse, there would also be a good argument for intentionally collapsing any skyscrapers that were still standing, as depressing as that is. An unmaintained skyscraper will be shedding glass panels fairly frequently, and once the window glass is gone it will start accumulating rainwater and corroding the structural steel. Uncontrolled collapses also have a risk of knocking down neighboring buildings. Walking through a downtown area 50 years after it was depopulated would be fairly dangerous. You would want crews stripping empty buildings and knocking them down asap.
Not to mention large cities are located at ports where you're going to want to operate anyway, once civilization is rebuilt enough to engage in ocean freight again.