I see a lot of commentary on break quality and stopping distance.

Those things matter, but not as much as reflex response time, which is mostly a fixed constant for most people.

Going faster means you travel further before even getting to the brakes.