> I'm pretty convinced most stress caused by driving is just people driving way too aggressively.

In my experience, the exact opposite seems to be the case. Aggressive drivers tend to move with a purpose, pay attention and react logically (even if recklessly) to events on the road. By contrast, the bored people zoning out while driving, the people stuck on their phones or just clogging up the fast lane while driving under the speed limit are unpredictable (and when they do wake up, they tend to panic) and cause problems all around themselves as everyone else is forced to adjust. People who can't be bothered to care about driving while doing it are much worse than people who care too much.