Something I know a boat load about. You are correct glass I would say is the worse for recycling (unless like you stated there's an infrastructure in place). At municipal recycling facilities (MRF) glass is the first to come off the recycling line. It gets smashed up and anything that is roughly 3inches or smaller ends up in the glass, so a lot of contamination.
Aluminum is easier to recycle, but you cannot melt down cans (UBC's) back into cans without blending metals as they consist of two different aluminum (the top and the body). If the cans are completely clean the big guys (alcan...) can melt them down. If there is any contamination it has to go to a secondary smelter which adds to the process.