They even have a specific callout for Comma:
> A determination was made to include the Comma Two Open Pilot system manufactured by > Comma.ai. Although Consumer Reports does not endorse after-market modifications to all > consumers, we feel that it is important to include the test results in this report to the industry. The > direct comparison of this system to the other OEM systems will hopefully provide insight on this > alternative approach and highlight the areas across the industry that have room for improvement.
It is at least SOMETHING. I haven't seen anything else even close to as formalized that looks across all the various systems out there. Do you have better ones to look at?