Sec.gov is blocking all requests from AWS?

1 pointatlasspring2 comments

I noticed sec.gov is blocking all requests from AWS since the 4 days ago. Has this been the same for you? How are you able to bypass this issue? Currently I can scrape sec.gov only from local computer. My ideal solution is that I can have a program that runs on some remote servers

View on HN

Add a user agent to the request

See here: https://www.sec.gov/os/accessing-edgar-data

You could use a proxy run on another provider, Linode, DigitalOcean, etc. Or you could scrape sec.report

What are you using SEC sites for, just EDGAR or something else?

Reply on news.ycombinator.com