The most annoying thing I encountered was the Switch controller support being rather poor. Every button press was somehow interpreted as two different buttons at the same time and I had to figure out which commands to run on Terminal to stop it from happening. Even then, the bluetooth connection on my PC was so bad that I had to stay within 3 feet lest the controller disconnects. I don't really think this is a Linux issue per se, but I recommend people buy a couple of 8bitdo controllers on Amazon which come with USB dongles if they want to go this route.
I will miss games that I can only play with mouse and keyboard, but I think there are enough games out there with controller support that this is not going to be an issue.