It's more complicated than that. Many companies that aren't mainly in tech end up having strange ideas of what competitive pay is for FTEs, thanks to HR, but managers can spend capital project money with far fewer limitations. This leads to places where the contractors are very well paid, but there's no way to convert them to FTEs, just because the offers are never high enough. Add to that the fact that raises in those places are rarely competitive with the market, and you can see crazy situations where a contractor is doing so much better than an FTE that has been in the company for 5 years.
I've also seen layoffs where only full time employees get hit, while the contractors, just don't get hit. Just this month, I've just seen a 20% cut in FTEs , followed by the remaining employees having to spend a lot of time interviewing contractors to replace them. And the contractors are not going to be on-call, so support rotations shrink.
So yes, there are places where the long term contractor is the least desirable position available, but it's really hard to generalize.