That's really a huge issue right now (to some extent even before the AI hype) that almost everywhere google is effectively the only entity explicitly allowed to scrape.