While a lot of metrics make intuitive sense, we don’t have that much hard evidence to prove or disprove their value. Part of it is the whole “if a metric becomes a target, it ceases to be a good metric” thing. Adding the checks to a large existing project probably has negative value. But I think it’s worth doing for greenfield projects.
For humans, these should just be advisory. But for LLMs I’m happy enough to make it a blocking check.
I keep thinking of doing an experiment where I give the same LLM the same problem, and only change which metric is enforced. And then see if any of them have a noticeable effect on correctness/maintainability.
> any examples of how people get this into an actual report / CI test / benchmark / whatever ?
Yeah they have examples of adding it to CI, or local checks, generate html reports, etc in their docs.