I am not sure we want to prevent AI crawlers but rather we want the crawlers to just not negatively affect the websites.
We want AI automation everywhere and crawling is important.
We want AI automation everywhere and crawling is important.
Who is we? I definitely don’t want AI automation everywhere