Hacker News Reader

What's stopping you, they crawl your site day and night and sell the data they have harvested at a vast profit. They don't even respect robots.txt.

AhrefsBot respects robots.txt. Details are here: https://ahrefs.com/robot

My parents say I am a very good boy too.

If you don't believe the CEO you can always just IP block them. In my experience they respect "User-agent: * Disallow: /".

Blocking the user agent in my WAF is the only thing that actually worked. So, I'm sure you're right and I am just a random outlying case.