Crawling the internet and dumping it to disk is not "stealing".
Mass downloading copyrighted works is. Which they did. Aaron got threatened with 20 years, they got pentagon contracts.
Is this stealing? Is it depriving NYT or publishers/writers from money via lost sales/subs? I don't know, but it certainly could be.
[0] https://www.nytimes.com/2023/12/27/business/media/new-york-t...