One I found from 2020 shows nginx can handle more RPS in less time for HTTP/2 and HTTP/3: https://caddy.community/t/caddy-v2-http-2-http-3-benchmarks/...
A proxy benchmark from 2021 also shows nginx perform better than Caddy: https://github.com/NickMRamirez/Proxy-Benchmarks
So yeah if you worry about peak throughput on a CPU-bound system Caddy probably isn't the right choice though I'm not sure nginx is, either. A hardware-based load balancer, at least as a pre-stage is probably more sane in those cases. For everything else I would always trade ease of usage for a bit of performance. I would e.g. never dare to touch the nginx codebase, but writing a plugin for Caddy doesn't intimidate me at all.
After testing my app with it in local dev I noticed what the benchmark above says: requests per second is about half Nginx. So i switched back to Nginx.