Thats the thing, LXC isn't a VM, its a container, so they all share the same kernel. I think its worth a shot, so I may test this.
FWIW Incus is really good here. I don't quite have it fully working for headless gaming containers sharing a single GPU, but I'm very confident it will work at this point.