Hacker News Reader

I wonder the same too, also is Gemini Pro supposed to beat GPT-4?

I thought so too but no, Gemini _Ultra_ is supposed to beat GPT-4. The one available through Bard is Gemini Pro. The wording sure is confusing!

Looking at the technical report [1], on selected benchmarks, rounded score in %:

Dataset | Gemini Ultra | Gemini Pro | GPT-4

MMLU | 90 | 79 | 87

BIG-Bench-Hard | 84 | 75 | 83

HellaSwag | 88 | 85 | 95

Natural2Code | 75 | 70 | 74

WMT23 | 74 | 72 | 74