Assuming a single gigabit symmetric connection could dedicate at most 100mb of upload bandwidth, you'd need one such viewer per each 25 viewers with a worse connection. This feels achievable.
You'd have 1 server that broadcasts to at most 10K tier-1 viewers. Tier 2 viewers get 3 tier1 ips for failover, and pre-negotiate a connection with them, e.g. through STUn, to get sub-second failovers in case their primary source quits the stream. A central control plane load balances these.
With something like 15s of buffer (acceptable for a gaming stream, not so much for sports, where your neighbors might be watching on satellite and cheer), this feels achievable.