Bufferbloat.
Ideally, at least for TCP applications, with perfect network congestion control, reduced bandwidth due to streaming should only degrade data rate, but never massively increase the latency. A slight increase is normal, but a huge increase is not. Unfortunately, in real life, it's often the case, and this aspect is often overlooked by vendors, developers, and sysadmins, making things even worse than it should be.
Network hardware or software is optimized for throughput, and its performance under heavy traffic is often not tested. One common practice is using a buffer that is as large as possible, so that you can push the data rate to the maximum and avoid dropping packets at all cost. This is called Bufferbloat and it's a latency disaster. When the upstream bandwidth is saturated, the downstream buffer keeps accepting more bytes/packets, effectively breaks the proper feedback signal used by congestion control algorithms, so it never kicks in in a timely manner, and the FIFO nature of the buffer means the latency of a saturated link is always as high as the buffer size. As long as something is using the bandwidth, the network will always be slow like a snail. When people start having this problem, they simply blame the bandwidth-intensive application, and set a QoS up to prioritize things like DNS and VoIP, and to punish downloaders (and big networks usually have incrediblely complex and elaborate rules), This appeared to "fix" the problem, but it does not solve the underlying problem, all it does is moving the problem to the low priority queue. Another trouble is unwanted buffer can exist everywhere, in applications, operation systems, and underlying hardware, and since there's a conflict between throughput and latency, and some buffers are even technically necessary (Wireless networks are the worst offender), there's still a long way to do to fix everything.
I am only speaking from my experience of managing small LANs, but I got most of the information from Dave Täht and Jim Gettys, who have been working on this issue in the last 10 years. Gettys writes extensively on bufferbloat in his blog that states similar issues exist at a much larger scale, including the ISP edge and backbone. Perhaps the vast majority of the case we are seeing here is caused by a lack of bandwidth and not related to bufferbloat, but I guess bufferbloat is still here to blame for at least 20% of the cases.
Here is a talk Dave Täht recently made: https://blog.apnic.net/2020/01/22/bufferbloat-may-be-solved-...
And here is Gettys' blogpost: https://gettys.wordpress.com/2018/02/11/the-blind-men-and-th...