The small amount of discomfort you experience from people driving too slowly is nothing compared to the impacts of people trying to drive as fast as possible. Turns out, that traffic paradoxically moves FASTER (in the aggregate) when you reduce speed limits.
https://theconversation.com/increasing-the-speed-limit-wont-...
https://eprints.soton.ac.uk/413955/
https://3659de2n61p72dta253nvqzd-wpengine.netdna-ssl.com/wp-...
https://3659de2n61p72dta253nvqzd-wpengine.netdna-ssl.com/wp-...
No, you do not need to pass slower traffic. Turn your radar cruise control on, enjoy your tunes, and learn to delay gratification.