The benchmarks aren't great, they're super specific to sem's output: why would I ask Claude how many "entities" were modified by a commit and do I need a tool specifically for this request ? Note that an "entity" is a sem-specific concept...
I can also give my thought process, because I was more interested in figuring out the model's inherent search results and understanding without sem.