As the Cloudflare post indicates, most crawlers can be verified by IP address.