For one-on-one videoconferencing meetings with normal latency (e.g. within 3-4 time zones), I find zero fatigue or problems like this.

The only "fatigue" I find is frustration when people are talking over each other or multiple people trying to jump in at the same time. This generally happens either when latency is bad (e.g. US-Japan), connections are bad generally (MS Office decides to auto-update in the background and saturate your download), or just when it's a particularly contentious meeting with lots of people trying to get a word in edgewise, and all the normal cues you use to figure out when to talk just aren't there. And the reasons for those all seem self-evident.