What's protected by math is the custody of the coins. The system is a delicate balancing act of incentives, math and social consensus. It's common among detractors to want to flippantly dismiss it as "only having value because of shared belief." That's not entirely accurate. All of the components are necessary to give Bitcoin its interesting properties which allow it to function as the first currency not managed by centralized control and these properties are inherently valuable.