I have done that with XReal glasses and my phone.

But they still look kind of off when wearing them in public (slightly bigger than normal sunglasses, cable from one ear and other people can see the light from the screens from the side or back).

And the lack of integration is a pain - the phone has to be unlocked so is subject to random taps and swipes in my pocket.

However, an Apple-built Carplay-style projection into XReal type glasses could work very well - the question being how would you control it?