Caddy does have some bizarre limitations I've run into, particularly logging with different permissions when it writes the file, so other processes like promtail can read the logs. With Caddy you cannot change them, it always writes with very restrictive permissions.

I find their docs also really hard to deal with, trying to figure out something that would be super simple on Nginx can be really difficult on Caddy, if it's outside the scope of 'normal stuff'

The other thing I really don't like is if you install via a package manager to get automated updates, you don't get any of the plugins. If you want plugins you have to build it yourself or use their build service, and you don't get automatic updates.