It's unusable even with the "user-friendly" distros like Mint and Ubuntu. Starting with the fact that Mint and Ubuntu don't even agree on what window system to use.
People do have difficulty switching between Mac and Windows, but each has critical mass so it's still easy to get help with the finer details. And unrelated to fragmentation, anyone tech literate won't have nearly as many showstopping issues to ask about there in the first place.