Would that be OK?
I ask because many laptops used to have similar setups.
the host never sees the card.
There was this recent improvement at least: https://phoronix.com/scan.php?page=news_item&px=Linux-5.14-A...
edit: seems there are some howtos - eg https://github.com/joeknock90/Single-GPU-Passthrough
1. Desktop environment, which uses the video card; this is not a big deal, since one can terminate the session
2. video drivers correctly and fully releasing the card; this is possible, but in my opinion, not stable.
I would say the use case here would be a machine that does not have a monitor connected, for example some Bitcoin mining server in the attic. With Looking Glass the game screen can get streamed to a ultrabook in your living room, with the server GPU performance