Barefoot or not your feet are dirty, the shoes are a different kind of dirty, and there's other stuff on the floor too (like in my instance, I really need to get a roomba because my cat sheds all over the place). So whatever.
I'm pretty amazed guests will take their shoes off because everyone that lives here does, like this is what happened when a whole bunch of people came over to my place after my wedding: https://www.flickr.com/photos/chix0r/13618218504/
Don't even feel like discussing how clean are fresh are everyone's socks.
Agree socks might not be 100% clean but you have to draw the line somewhere, and the bottom of a sock is bound to be cleaner than the bottom of a shoe.
In central Europe it's common to have a special room called checkroom/garderobe next to your main door where you store several pairs of guest slippers.