Buses have to let people off, as well as on, no?
The faster bus skipping stops where people are waiting, so they can be picked up by the next bus does work; I've seen it in action.
Most bus stops I have used in the US dont have covered shelters or enough space for passengers to wait outside of the elements. Usually this means sitting under the overhang of a nearby business. If the bus doesn't want to stop for whatever reason and blows by, it sucks.
Maybe a better future would be a public bus app or button at the stop that I can press to say I am waiting, but until that day, they should stop.