Are you seriously arguing that sleeping out in the nature, potentially with dangerous animals around you, on bear skins, with several friends (on the same bear skin?) is as comfortable as sleeping on a modern mattress in your home?
I think about it in terms of germs. Yes, on paper it's more safe to sanitize everything, but in doing so we prohibit our ability to build defenses, and actually become less healthy.
In the same vein, maybe by sleeping on mattresses indoors all the time, we fail to build a tolerance to adverse sleeping conditions, and maybe get more sensitive to them too.
So if your body adjusts to it, it's definitely possible to actually prefer it.
The worst is when you have an uneven surface, like when camping and there is a rock which even your (inflatable) roll mat cant dampen out, that makes sleep difficult.
The hard ground isn't giving either, but you feel it. ;)
I also can't deal with pillows when I have a migraine. Though I can't sleep without something under my head. I usually just roll up a towel or something. As I said before, I think it's about having precise control over body position.
Hunter gatherers thriving on 5-7 hours aren't sleeping on modern mattresses either.