Hacker News Reader

Comment by paulgb | Hacker News Reader

Google Sheets can be surprisingly handy as a UI for editing data. For example, I have a job on AWS that scrapes a handful of URLs and snapshots them. Instead of creating a database and hosting it somewhere, or hard-coding them, I put them in a spreadsheet that the AWS job reads at the beginning of every run.

View on HN

I just made an example for how to do this using Refinery (most of the pieces were off the shelf using "saved blocks").

https://app.refinery.io/import?q=9lvohdvqd50g

This project will:

- Run every hour (via timer)

- Grab URLs from a Google Sheet

- Go screenshot all of the URLs

- Write each screenshot to an S3 bucket

- Email you at the end after the jobs finish

There is more I could do with this (like trigger an email if something fails, etc). But at least does what you describe above in a (hopefully) succinct fashion!

Also, shill alert as I'm an author for this service (but this was a fun project to build over a few minutes, so thank you).

Aren't you forced to use Google Apps Script to interact with Sheets? I remember being forced to use it for interacting with Google Forms API.

> Aren't you forced to use Google Apps Script to interact with Sheets?

No, it has a REST API; you need Apps Script if you want automation within Sheets, but you can interact with Sheets without it.

That's the best part, I don't even use an API! I just go File -> Publish to the web, then select "Comma-separated values" instead of "Web page" from the dropdown. Then I use Pandas to read the CSV in Python.

I continually find myself amazed with what people can do with Excel.

haha yeah and I always wonder why especially engineers are so creative in using Excel as a graphics/painting tool :) Looks good, but a nightmare to read in a programmatic way ^^

How do you get around the number of fetch limitations?

I'm hitting at a rate of 2/hour, it hasn't been an issue.

how are you accomplishing that? Does Google sheets have a scheduler built into it? Long running timer?

The scheduled job runs on AWS, it just fetches the sheet when it runs.

Google Apps Script has a scheduler built in: https://developers.google.com/apps-script/reference/script/c...

I use this to automate a lot of tasks. E.g. automatically clip supermarket coupons every week, check for tax loss harvesting opportunities in my portfolio, etc.

It's really powerful

Reply on news.ycombinator.com