I say it is more akin to shipping produce than TVs. We don't get the majority of our veggies from a single farm for all of the US.
People want to spend on mattresses because they believe spending a lot on mattresses will improve their lives in some way. Less fatigue, less pain, less stress, etc. These are big selling points. However, they are selling mostly snake oil.
As for why mattresses are so expensive, I think a lot of it is "because they can." I think there is also materials and such. A bed with two people in it might hold over 400 pounds for 8 hours a night 7 days a week for years. If a tv had to do that it might be more expensive too.
I lean toward the 'because they can' argument, which means, what the market will bear.