If this were a limitation of a free plan, with a paid option offering higher bandwidth, that would be cool, but the only paid options they offer are app integrations starting at $100/mo for 300 users.
It's not inevitable. I use Jitsi every week for a 3+ hour meeting and I haven't seen that error in more than a year.
I don't remember if I ever tracked down what was causing it, but it may have been high latency or low bandwidth on my end (which I resolved about a year ago).
Worked.
Most people don't have a network at home that can handle, say, sending 50 copies of your AV stream when you're speaking to 50 other participants. With that many, even audio-only is a challenge.
There are some clever p2p relay mechanisms you could use in theory to bring the bandwidth down to not grow as much with number of participants (but still a constant factor more than the hub method), but they would add significant latency. For example your AV is sent to one peer, they send it to the next peer, and so on in a chain or tree. This works well for non-real-time AV like p2p distribution of movies, but not real-time meetings.
If multicast IP worked over the internet at large that would help but it doesn't work. Even if it did it wouldn't be enough for the full-vs-thumbnails continuous renegotiation needed to keep download bandwidth steady, nor could it do per-user quality-vs-rate adaptation.