He's a really smart guy, and friendly, but his heads-up display came across as very distracting to me, mostly because of how it interfered with "intuitive" reading of eye gaze. When he was looking vaguely in my direction it was difficult to tell when he was looking "through" the heads-up display and focusing on me, and when he was focusing instead on the HUD. It made the whole conversation feel quite surreal.
Glass is hardly the problem there - if the conversation is interesting, people will turn off any music being played and ignore whatever is happening on Glass.
Not necessarily. People can be very distracted by cell phones which are normally kept in one's pocket or purse. Having glass on your face with visual pop-ups would be even more distracting, regardless of the conversation, because it would be a lot harder to ignore incoming alerts or other cues.