I see the same difficulties with some games, for instance those that do not lend themselves to a simple score board.

When I join a stream of someone playing a strategy game which also has a tactical layer, you are usually equally clueless as to what is going on. Certainly at the strategic layer but also as to what a good outcome of a tactical encounter is.

I'm not sure how technology can fix this. On the other hand, other spectators can usually pretty easily do this in chat.