Yesterday, Anthropic released its latest model, Claude3, which has attracted widespread attention and praise for its outstanding performance. In the LLMRGB project of the open-source evaluation project on Babel.cloud, Claude3 scored a high score of 97.6 in a single test, greatly surpassing GPT-4 and becoming the current leader in large model capabilities.
Notably, in the LLM-RGB evaluation, 015_simple_mahjong is an extremely complex problem. In the prompt, the simplified rules of Mahjong are taught to the large model, with examples provided, and then the large model is asked to make a play choice in a specific scenario. This problem has rarely been correctly solved in past tests. However, Claude 3 Opus has a 20% probability of giving the optimal solution and an 80% probability of providing the suboptimal solution. This indicates that its multi-round reasoning ability far exceeds other models, and it can quickly learn and apply knowledge with limited context. This will make the application scenarios of Claude 3 far more than simple customer service and text generation. It can perform well in fields with longer engineering processes.