The only thing that really annoys me about this is that the resources get scraped more than once. Sure, update a delta every month or whatever, but can't the AI bros get the sh*t together and share scrapes? It's embarrassing.

The special sauce is in parsing, tokenizing, enriching etc. There is no value in re-scraping, and massive cost, right?

Just do it.