There shouldn't be any significant virtualization penalty. My guess is that it uses hardware video codec acceleration and that wasn't accessible in the VM you had set up? If the guest was Windows, looks like nvidia has supported this since host driver version v465 or later.
https://nvidia.custhelp.com/app/answers/detail/a_id/5173/~/g...
HTH!
The security problem is being able to talk to the same X server as trusted applications. X clients can do pretty much all the things you don't want Zoom to do; look at your screen, observe your keystrokes, etc. (Sadly, many of Zoom's features, like screen sharing, are also great things for spyware to do in the background. Not saying Zoom does this, but if you don't trust them, this level of access is the part that worries people, not consuming too much CPU.)
Use Ctrl+Alt+F<number> to switch into another VT and run a different X server. Run zoom in container there.
I found this a lot more convenient than messing with nested X servers and other types of X11 client isolation. Each time you leave an X server and switch to another VT, the clients perceive it like the monitor being turned on/off.