Two hosts on the same LAN can have excellent performance with VNC. There are some things you need to know and some work you need to do, however.
First, the two most common DEs, Gnome and KDE, use X compositing by default. This is very bad for VNC. You want to turn that off. Unfortunately you can't with Gnome, which is sad. You can turn off compositing in KDE, and that's one reason I've preferred KDE for years. Other DEs can also forego compositing.
Second, turn off all encryption between the VNC client and server. No SSH-ing, no build-in VNC encryption. Nothing. Ignore the nag warning about the lack of encryption some VNC clients hit you with.
Third, no VNC compression. You're on a LAN and you don't need it or the lag it adds.
Forth, don't "share" the VNC session with the remote host's desktop. Use the vncserver headless X server. Every "shared" VNC system (where the desktop appears both local and remote) I've encountered is laggy.
Fifth, use a fast VNC client. RealVNC's VNC Viewer is excellent.
Sixth, do not scale. Ensure your vncserver's resolution matches your VNC's client size exactly.
Seventh, tune your DE to remove whatever animations or alpha effects exist.
Eighth, no wifi. Ethernet only between VNC server and client. Wifi is awful for this.
Ninth, use decent ethernet gear. Some NICs are low end and impose more latency. Likewise, low cost switches are bad news as well.
Do these things and your VNC session will be difficult to distinguish from a local DE. If you're using gigabit or faster LAN you'll be able to play video through VNC reasonably well.