This is poor architecture, not Lambda's fault.

Same result would be achieved by paying tons of $$ for a high-end machine with dozens of cores, just to scrape one URL per core.

They could have implemented asyncIO within just a few Lambdas - heck even one Lambda could certainly handle hundreds, if not thousands of ASYNC jobs at once, just as a cheap machine, as you pointed.

They treated Lambda functions as threads or async jobs, while they should be looked at as core processors.