Yeah, the state of the art is reverse DNS and then checking that the forward DNS matches which is quite a mess and requires careful use of egress IPs and depends on the network for security. Actually signing requests is a huge improvement.

And while Cloudflare wants them to register which isn't great the standard does allow automatic discovery and verification of the signing keys which allows you to reliably get an associated domain which is very nice.

As the Cloudflare post indicates, most crawlers can be verified by IP address.