Thank you! This is very interesting.

I have wanted to implement something like this one - ie: Lambda doing the downloading of the page itself.

I have wondered how it would work with very strict sites like Yelp - limits similar to what you would get in their API (so doesn't make sense not to use their API).

What are your stats like if you don't mind? How much people are using it and much are getting blocked (404 or 500 after 1000 requests, etc.)?

Edit: Is it possible to use my own credentials for AWS?