Most is purchased with debt - this is what sets the price. Also they are not making new land. Why should people have the right to charge people to live on a part of the earth in perpetuity? The state builds infra around it then the land owner charges a fortune for people to have the right to exist.
Land is a special case and should be treated as such. Land value tax would address this by capturing most of the added value from land for the common wealth rather than land monopolists.