This is my experience as well. I currently use a USB switch that I got off of Amazon for about 15 bucks. It typically "just works." Press the button, wait 2 seconds and my input devices have switched.
I use it to switch all of my peripherals (keyboard, mouse, wireless headset dongle, webcam, and mixer) between my desktop and my MacBook several times per day and have not had a single failure yet.
I've used it with Windows 10, MacOS and an RPi, with no issues.