That’s what I did (and with one USB cable swap and a button press on the monitor, I can switch keyboard, mouse, monitor, speakers, and camera between them).

But the key for me is I do that because I have a work laptop and a personal desktop and I want no question about the ownership of things I do on the desktop.