Biggest complaint by far was trackpad support in Linux. It's a mess out of the box: weird acceleration curves, bad scrolling, twitchy, overly sensitive clicking, the works.
Luckily it's Linux, so it's Configurable™, and with enough tweaking I had a script that would feed a bunch of settings into the userspace synclient synaptics configuration tool, and get a trackpad experience that was mostly bearable. EXCEPT that the damn driver would somehow reset itself randomly every hour or two, and my trackpad would be back to the default mess until the script re-ran.
There were the other common problems too: an update broke X config, font rendering was janky, battery life was a joke. The Alfred replacement I was using would forget its keybinding on every startup. Unity/compiz would stop respecting screen hotcorner triggers until I open the config, cleared, saved, reenabled, and resaved them. Etc, etc, et-fucking-cetera.
It's more or less the same experience I've had on-and-off with Linux on the desktop since 1999. It mostly "works", but it's inevitably death by a 1000 papercuts and after a few months I'm fed up with it and switch to something less grating.