Fun fact: macOS adds a default delay to the CapsLock to prevent this. Quick tap will not enable CapsLock. Only long tap.
Also fun: when remapping the CapsLock to control key using default macOS tools this behaviour remains. Eg: Mapping it to ctrl and trying to exit a shell command with ctrl+c only works with a long press. Super annoying.
Like all things on macOS you need a third party tool, like carabiner.