Assuming that your heads-up live display contains a public facing camera here's: Why not!
I don't like sitting in a bar (or a strip club, or in any other semi-public, or even public dodgy or non-dodgy place) and you hang there with your heads-up live display recording my image.
I don't like the fact that it's uploaded to some tech behemoth' server for further analysis (or even if not - by you later to your personal social media page, controlled by such a company).
I don't like an army of people like you recording every little nuance of my life, which then gets abused to feed even more AI algorithms by companies whom's major raison d'être is to violate all our privacy and infringing on my personal rights without permission or consent.
And no! Locking myself permanently into a room is really not a solution conductive to me or for us as a society.
Remember Google glass and why it failed? Not because the tech couldn't be useful. But it failed thanks to a company, which didn't give a shit about the impact of such tech and an army of glassholes (here's a nice example[1]) who cared even less.
I think the answer to your "why not" and to your personal gratification is that the price for us as a society is far too expensive.
[1] https://www.google.com/search?q=robert+scoble+glass+hole