Also I've checked their docs to see if there is any mention about the user agents or IP ranges they use for scraping, with no luck.
It does respect robots.txt when crawling. I'll add more details about this in the docs.
I appreciate the reply. As someone who runs multiple CMSs it’s painful to deal with the ai crawlers these days. Specially the ones that don’t respect my terms.