Interesting that 4.7 failed like that. Seems 5.5 is impressive but is oh so expensive.
Would be interesting if you ran your same test with Deepseek v4 and some of the other Chinese models.
Would be interesting if you ran your same test with Deepseek v4 and some of the other Chinese models.
I also tried GLM 5.1, it's first attempt was such a disaster I didn't bother working with it any further. It also took by far the longest and wasted a bunch of time/tokens trying to find other converters online (and failing) instead of just reverse engineering the format from the sample files given.
the thing is I've tried to come up with a good test my own and spend countless time just tweaking it instead of saying this is good enough and benchmarking.