Where did they say this?

I don't recall them saying that, but, I mean, is Gemini Ultra a "ton" better than GPT-4? It seemingly doesn't represent a radical change. I don't see any claim that it's using revolutionary new methods.

At best Gemini seems to be a significant incremental improvement. Which is welcome, and I'm glad for the competition, but to significantly increase the applicability of of these models to real problems I expect that we'll need new breakthrough techniques that allow better control over behavior, practically eliminate hallucinations, enable both short-term and long-term memory separate from the context window, allow adaptive "thinking" time per output token for hard problems, etc.

Current methods like CoT based around manipulating prompts are cool but I don't think that the long term future of these models is to do all of their internal thinking, memory, etc in the form of text.

isnt that wrt scaling size? couldn't they make other improvements?

i'd be real interested if they can rebut with big multimodal improvements.

It just has to be good as old gpt-4.

I don’t think that’s the case.