Gemini Benchmark – MMLU (compared with GPT-4-turbo, Mixtral)

Most interesting is how they perform by subtask. E.g. gpt-4-turbo vs gpt-4-turbo with chain of thought (cot) goes from 0.51 to 0.88, but sometimes cot makes things worse (e.g. in management).