Same problem: The models are also correlated on what they can and can't solve.
To give you an extreme example, I can ask 1000000 different models for a counterexample to the 3n + 1 problem, and all will get it wrong.
To give you an extreme example, I can ask 1000000 different models for a counterexample to the 3n + 1 problem, and all will get it wrong.
For reference: https://en.wikipedia.org/wiki/Collatz_conjecture
So no, sampling 1000000 LLMs will not get you a solution to it. I guarantee you that.
If you want to scale back, many programming problems are going to be like this, too. Failure points of different models are correlated as much as failure points during sampling are correlated. You only gain information from repeated trials when those trials are uncorrelated, and sampling multiple LLMs is still correlated.
Is that the correct answer to "write a lock-free MPMC queue"? That is a coding problem that literally every LLM gets wrong, but has several well-known solutions.
There's merit to "I don't know" as a solution, but a lot of the knowledge encoded in LLMs is correlated with other LLMs, so more sampling isn't going to get rid of all the "I don't knows."