Experience from political betting spaces tells me that the wisdom of the crowd rarely actually holds up to scrutiny. Political bettors[0] hardly ever do as good as simple poll aggregators like 538 or RCP.
Also, the video you posted points out a critical part of it working: diversity
I think that's why an aggregation of independent pollsters, as few of them as there might be, often ends up giving us a more accurate prediction than the millions of users that engage in PredictIt or similar betting websites