If you're worried about that, have house shoes (or slippers). This is the norm, especially providing slippers for guests, in no-shoes homes.