eckr

"Claude Mythos 5.1 is identical to Fable 5.1, but it offers more permissive safeguards for vetted individuals and organizations"

Then why does it have separate datapoints for Terminal Bench, and score higher? Something doesn't add up here??

Maybe this is just my experience, but have people had trouble with 3.6 Flash just... getting things it has seen in its context correct? I don't know if it's been insanely benchmaxxed or what, but it'll pull information from websites and immediately get it wrong the token after. Or for example (this is something that happened like yesterday) I asked it to compare the uses of A and B in a language I was learning, and the way I typed it was "Please compare how these two are compared differently: A VS B", and then... it proceeded to compare "VS" and "B". I'm not kidding.

Personally whenever I use Gemini I've just been using 3.1 Pro because I've had insane trouble with them getting things incorrect like this. Hopefully they'll fix it soon / they've fixed it with 3.7 Flash.

There was a startup that did this for Llama 3, I forgot their name. Etched is also doing some similar things I believe.

Obviously it comes down to that, but you can't make the claim that GPUs aren't a huge part of it. Otherwise, billions wouldn't be getting invested into them in the west, no? And "financial backing" essentially boils down to the researchers, which boils down to the quality of the data and research, and to compute. I do think they have really smart researchers, obviously — otherwise this wouldn't be possible.

This is fair, with the caveat that we don't know for how long this model has been around internally in China either. So we can only go about appearance / releases.

This, essentially. Especially in compute I don't think it's debatable they have less resources than the frontier labs in the west.

These benchmark numbers are insane. The days when China was 6 months behind are over? How are they doing this with so much less resources than the US??? I have so much respect for the researchers there

Because, realistically that's all programmers ever need, would be my guess. I do think linear algebra is an extremely interesting topic in its own right / outside, but yeah.

In the past, they just ran Deepseek OCR on your image and extracted the text, then gave it to a language only model. I believe now there is a model that actually takes images as input directly.

I don't think this is private knowledge guessing from when and how I was told, so I feel comfortable sharing it. When I talked to some Huawei representatives, I was told DeepSeek V4 was trained entirely on Huawei chips. It's up to you whether you believe it or not, and while I see the incentives in faking these news, the blow if not true would be so massive that I don't think their representatives at large venues would be making these claims without thinking it's truly correct.