Aren't swamp coolers only really useful when the humidity is under something like 50%? I guess I'm assuming you're talking about the US, but I don't think "vast majority" of people live in such areas. A quick search for average humidity of US states in July and August shows many of them have an average daily humidity of over 50%.
It certainly rules out every place I've lived on the east coast (Richmond, Virginia, Long Island, and the Boston area), as the humidity in those places is usually at least 60% at the height of summer when most people actually use their AC.
As a side note, I suspect most people overuse their air conditioning and don't let their bodies adapt to the warmer summer weather. It's weird how many people keep their houses under 75 °F when the heat index is only in the mid 80s.
They are less effective, but still work. As long as the dew point is ~72 degrees, or lower, they can still cool enough to do the job (although of course not to the extent of an AC). But yes, ultimately it comes down to people being ok with not setting the AC to 68 degrees when it's 80 degrees out, and using the minimal amount of cooling necessary to be comfortable.
You can use this tool to visualize this: https://drajmarsh.bitbucket.io/psychro-chart2d.html
(Use the Givoni Bioclimatic Chart overlay and select a weather station using the globe icon.)
That said, there are a number of more complex devices which could further reduce how often AC systems need to be used: dew-point (Maisotsenko Cycle) coolers, indirect evaporation cooling, and thermal energy storage to name a few.