Perception: Linux has historically required tweaking and doesn't work straight out of the box. This requires computer skills, and the guts to actually make changes that might potentially break your installed system. For the mainstream, this is a turnoff. While things tend to work better today, the image persists; Linux is for geeks, and is too difficult for people with no interest in the computer itself.
Software: While tools such as wine, playonlinux, steam and similar alternatives make life easier for those who want windows-only applications in Linux, they are not perfect and rarely work out of the box, especially on new applications. While patches or workarounds tend to turn up, around half of the applications on appdb.winehq.org are rated silver or less, requiring tweaks to work, if at all. People want their things to work without any hassle, and aren't interested in spending time to learn how to configure wine or the rest of their system for it to work.
Hardware support: Although improving, drivers can still be a problem. Getting drivers from eg. Nvidia is fairly easy but installing them is not as easy as it is for Windows, unless you know how to add the Nvidia PPA (which requires knowledge, see above). AMD is supposedly worse, but I have still to buy a GPU from them. Getting drivers for other proprietary hardware can be a lot more difficult, depending on what it is. I've got a fairly common MB from MSI, but I still have to tweak the default Ubuntu config to get the sound to work. If it wasn't for the fact that I know how and prefer Linux over Windows, I'd just stick with Windows (where it just works).
Why bother: There is nothing Linux does so much better for most people that it's worth taking the time to switch. Most people I know (not counting colleagues) know virtually _nothing_ about their computers or how they work, let alone which version of Windows they run. I think most people are somewhat like that; the computer is a tool for them to do "stuff", as long as it works, they're happy. Or at least content. Switching to Linux doesn't help them in any way, but generally mean that they have to figure out how to get a lot of things working properly (which used to work just fine in Windows) and they have to struggle with howtos that might be outdated or just plain wrong for their situation.
A major selling point for many Linux advocates is that it's free and open source. But most people doesn't care; they might buy a new laptop every 3-5 years which comes with Windows pre-installed, along with a discount for Microsoft Office and a bunch of other stuff they think they need. So why bother having to learn something they don't see the benefit from?
Once you look at the post-desktop world, Linux is mainstream. The cloud, mobile, IoT devices. Linux has very large market-share in all of these.