Even at orders of magnitude greater speed, we've still hit diminishing returns for quality of output. We simply haven't found anything like superhuman reasoning ability, just superhuman (potentially) reasoning speed.
All the easily verifiable domains such as mathematics, coding, and things that can be run inside a reasonable simulation are falling very very fast.
By next year if not sooner, mathematicians will be wildly outpaced by LLMs for reasoning.
Only if you fully detail the behavior of the system.... at that point why use a chatbot? You've coded the entire thing.
> first as good as human
We'll see. Chatbots are only as capable as you detail them to be
> research showing with the right policy, Rest of the owl.
So it's not impossible to have things that seem orthogonal, like generation speed or context length, have an impact on quality of result.