750k items in their content management sysem. N independent labs crawling wanting to check that every day could easily give bursts of millions per hour
Millions per hour is tens per second though; perhaps the fix is performance improvements
Millions per hour is tens per second though; perhaps the fix is performance improvements
That'll be great until.. they rewrite the scrapers in Rust! Then we're really hosed!