As someone who occasionally streams, max latency would be around a consistent 7-10 seconds - that makes some interaction patterns (the quick "which choice do I pick?") harder but still allows some reasonable level of feedback.
Part of the reason I personally watch streams is to feel connected to the rest of the audience - when something good happens we all cheer immediately, when something bad happens there's immediate commentary. Being split up from part of the audience would cause me to feel weird - as one of the vocal minority I'd be cheering in a much smaller crowd, and as one of the passive viewers I'd always be wondering what the vocal minority were on about.
There's also another type of stream that's popped up on reddit's RPAN - just random people chatting and interacting with their audience. That's not really feasible in Peertube's model.
Note that there's some types of streams that Peertube's model could be great for - pre-planned event shows without much interactivity, live captures of current events, and so on. GDQ, various music and performance events, broadcasts from protest zones, and so on could use this.