Not really. Having all that gear on just one side looks kinda cyborg-y. Symmetric is more natural.

Also, think of the possibilities for crazy 3D stuff if you have displays on both eyes.

why would that be a stupid idea?