That being said you cannot compare a model with a specialised harness. These are two completely different things.
Am I missing something?
That being said you cannot compare a model with a specialised harness. These are two completely different things.
Am I missing something?
> To increase efficiency, instead of processing literally every file for each software project that we evaluate, we first ask Claude to rank how likely each file in the project is to have interesting bugs on a scale of 1 to 5. A file ranked “1” has nothing at all that could contain a vulnerability (for instance, it might just define some constants). Conversely, a file ranked “5” might take raw data from the Internet and parse it, or it might handle user authentication. We start Claude on the files most likely to have bugs and go down the list in order of priority.
So they process it in parallel, but AI-rank them based on vuln-likelihood first = exhaustive search with a heuristic filtering first