HNHacker News
TopNewBestAskShowJobs

ep123

28 karma · joined February 21, 2017

submissionscomments
ep123··on Show HN: Getsy – browser/client-side web scraper
I added support for infinite scrolling sites.

Example here: http://www.getgetsy.com

ep123··on Show HN: Getsy – browser/client-side web scraper
https://github.com/epiqueras/getsy

I'm updating it soon with new methods that support websites with pagination.

ep123··on Show HN: Getsy – browser/client-side web scraper
I added a github page with a repl so you can try it out: https://epiqueras.github.io/getsy/
ep123··on Show HN: Getsy – browser/client-side web scraper
This tool will give you all the html that loads on your first visit to the url. You can insert it in a div's innerHTML, although this would be dangerous since all their scripts would run on your site.

You can get the raw response by accessing the Getsy's .content property.

ep123··on Show HN: Getsy – browser/client-side web scraper
crossorigin.me is a reverse proxy that will make requests for you and add CORS headers to them. You can specify the API endpoint of a proxy you want to use or let it default to crossorigin.me.