Insurance is simply the act of diversifying risk and normalizing returns. When you are insured, you are consequently
assured (confident) that you will not receive some huge, unexpected cost but that you will rather pay out in manageable portions over time.
It could be across a pool of investors, as you describe in the spice-trading example. Nobody wants to self-insure because of the chance of a huge catastrophe taking out their ship and netting them a huge loss and likely a lifetime of debt.
It could be across a pool of investors, as I describe in the health-insurance example. Nobody wants to self-insure because of the chance of a huge catastrophe putting them in the ER and netting them a huge loss and likely a lifetime of debt.
The reason we leave it to big organizations currently is because they have the resources (data, underwriters) to accurately asses and price risk. They charge a fee to do this. They are allowed a certain amount of extortionate leeway due to government regulation mandating certain minimum amounts of insurance, which you allude to in arguing with companies for reduced rates.
Things we could do to make the system better:
1) Reduce barriers of entry for doctors
2) Stop prescribing unnecessary treatments/prescriptions
3) Better education regarding health choices
4) More taxes/tradeable permit system for companies that pollute bodies or the environment (McDonalds, ADM, Dow Chemical, Bayer)