You're conflating two islands here -- Sea Island is a separate island that has been privately owned since at least the 1920s. You have to drive across St Simons to get to Sea Island. For sure, Sea Island is a high end, gated resort community of a few hundred people.
St Simons is home to about 12k people. It has been an expensive place to live since at least the 80s. I, too, share your concerns about the current development going on -- I'm not sure the developers are in fact being careful about killing the golden goose. Developers are cutting down our historic oak trees and indiscriminately putting up cheaply built condos as fast as the corrupt planning commission will let them.
This is the perfect opportunity to plug the St Simons Land Trust: https://www.sslt.org/