Show HN: Turn a Google Spreadsheet into an API | Hacker News Reader

Show HN: Turn a Google Spreadsheet into an API

sheetsu.com

357 pointsmichaeloblak98 comments

I've used Google Spreadsheet as an "API" before (manually, not using this wrapper), and it is definitely not made to be used as such. Under heavy load, it slows down or times out, and even under the best circumstances it's not exactly fast.

It's great in a pinch. My use-case was a spreadsheet that others in my organization wanted to edit, then we'd pull the data onto a web page and display nicely. It worked, but eventually we had to set up a secondary server to cache the spreadsheet result because pulling directly from Google in realtime was too unreliable.

Thanks for sharing. It's clear this approach works for small datasets and if you want something more reliable under heavy load just go for a proper database. Might work well with prototypes though.

One thing you can do to make this use-case work better is to combine the spreadsheet as API with a static site generator.

I made this very basic plugin for Jekyll:

https://github.com/netlify/jekyll-gdrive

That'll let you use a Google spreadsheet as a datasource and expose the data to your liquid templates.

Combine it with something like https://www.netlify.com that'll run the builds for you and let you trigger builds via webhooks, and you have a Google spreadsheet based publishing engine where the final sites are completely static and live straight on a CDN.

Used it for one site where we used the scripting options of Google spreadsheets to add a "Publish Now" image button to the sheet, so in the end user could edit the the sheet, and then press "Publish" to trigger a build+deploy.

If the users are already used to spredsheets, it can be a really handy little anti CMS :)

[disclaimer: I'm a founder of netlify]

That's a great use case!

Hey @bobfunk how do i contact you concerning your app - netlify.

Really great use case

You can shoot me a mail at matt@netlify.com

I also implemented something like this very recently [1] and I would recommend building in caching from the start as I was consistently getting throttled by the Google API even in development (just live reloading the page as I styled it).

Secondly, while the load page and then load the data in as JS approach definitely works, I found that just pushing the data in as HTML on the server side was easier to cache and faster to load.

1 - https://gist.github.com/mbuckbee/0ad3bd150e705c769c50

This was sufficient to handle thousands of requests per hour with a page load time of less than 250ms

Admittedly naive question: if the advantage of having Google Sheets store the data is that you can use the Sheets editor when you want to, could something like Sheetsu store the data separately, much faster, for the API, and then just sync any changes between the Google Spreadsheet and their data? So you could still access via Google when needed, but these problems don't matter?

I made a node.js package that does exactly this, a local cache in json with few hooks for data modification, and you can control when to update it to avoid timeouts [1]

Then we used it as backens for a blog for my University team as we were already using Drive, being each entry a google docs document [2]

[1] https://github.com/franciscop/drive-db [2] http://www.makersupv.com/

There are a bunch of 'fork-n-go' github jekyl sample applications that wrap a webpage with a Google Spreadsheet datasource.

Here is a sample (from a github employee) that I've used in the past. Source: https://github.com/jlord/hack-spots and live site: jlord.us/hack-spots/#info - not sure how they are caching the spreadsheet, but I've never seen it not perform (not sure about concurrent users or whatever though).

I've been looking at workflows like this, but the problem is being completely cut off from editing or testing locally when offline.

That is, it would be great to keep my data in a google sheet and transform it into a static json file that lives in my app source, but then if I need to work offline, all I can do is edit the json and remember to reflect the changes into the sheet later.

I don't suppose anyone has found a workflow that solves this?

We had a similar experience. Google started giving us "Sharing quota for this file has been exceeded" errors.

Cloudstitch is another service that gives you a RESTful interface over a Google Sheet. http://www.cloudstitch.com/

I made a video showing how to send a Raspberry Pi's temperature sensor data to a Google Sheet using Cloudstitch. https://www.youtube.com/watch?v=Cqa9Zkm7pCU

Cloudstitch is pretty neat for low-tech workers too. Reminds me of the same audience that Zapier targets. Great for marketing, report prep, etc.

Google Spreadsheets is awesome as a quick and dirty CMS. I use it all the time at the newspaper i work for, in combination with Tabletop (https://github.com/jsoma/tabletop). Only thing i lack right now is some kind of library that could cache those sheets using Redis for speed.

Flatware was built to be an S3 caching buddy for Tabletop, might prove useful for you: https://github.com/jsoma/flatware

That's nice - I've recently made a small project to wrap Github's Gists around an open API[1] for the same use case - a small open database easily browseable -, but your project seems much more interesting - spreadsheets are a very interesting way to solve this. Good job.

[1] https://github.com/arcanis/gist-proxy-server

Whoa, gists as a DB - that's a neat idea!

I would really prefer something like this to be a local library or something, adding another remote service in between negatively affects security and reliability. Especially in this case where there is no HTTPS yet.

Will do HTTPS.

Sort of related (but the other way around): https://www.blockspring.com/

Yes, Blockspring is really great software. I've one API endpoint (from Sheetsu) which is getting data via Blockspring.

https://etrading.wordpress.com/2015/09/10/blockspring-sympho...

1. How to get columns?

2. Your docs say: http://sheetsu.com/apis/12345/column/:column_name

Is ':' a documentation convention somewhere that I've not come across? I tried:

    http://sheetsu.com/apis/12345/column/:Email

    http://sheetsu.com/apis/12345/column/:email

until I realised it was just:

    http://sheetsu.com/apis/12345/column/Email

Yes, it's just /column/Email.

This is typically the way it's done in Node.js's Express library: http://expressjs.com/api.html#req

':' is not that uncommon. E.g.: http://mojolicio.us/perldoc/Mojolicious/Guides/Routing#Stand...

the colon is meant to show/explain the following is a variable

Would be nice you could get single 'row' too, by number like: '<url>/row/2'

Used this approach at a previous company and was amazed at first. Wondered why it wasn't more commonly used. A poor mans API! Whats not to like?? Then I found out why: when put under any kind of load it completely falls over and stops sending data at all. Also randomly would throw security/authentication errors that none of our devs noticed because it only happened on windows and older versions of OSX (we were a Linux shop). Its a fun and novel approach but should never be used in a production application that needs to scale.

Cool. I used https://github.com/jlord/sheetsee for a little thing. These things can be very handy.

What is the advantage of using this service over hitting Google's RESTful Spreadsheet API directly?

https://developers.google.com/google-apps/spreadsheets

It's easier.

Could you explain what you mean by easier?

You don't have to implement Google API or use any lib to talk with it. Just paste link and get API end-point with docs, which is easier to use. Much more easier than creating your app at Google Console, connecting with it, etc. Especially if you are not a programmer.

I recently hacked something together that's using Tabletop and React to render 'products' that can be entered into a 'database' using Google Forms:

http://jsbin.com/zaberiqami/1/edit?js,output

The top part of the code is separated from the bottom plumbing, and is sprinkled with comments in Dutch for my students to edit ('8th grade', Dutch 2VWO).

I actually used it more as an example and inspiration to talk about databases in general. The group is a mix of students from the previous year; by setting the bar high and encouraging them to change the parts they knew something about, I could gauge their individual skill level a bit. Pink product listings ensued. JSBin really is an awesome tool I couldn't do without.

Great My blog[1] uses a static side generator called middleman[2] and I was looking for a way to share 'quotes' in the website's subtitle. This seems to be the best solution! I was about to write a simple Sinatra API to run in localhost, but that's a way better solution for my use case!

Thanks!

[1] http://www.convalesco.org/

[2] https://middlemanapp.com/

UPDATE: A good use case is to use the HTML form to store subscribe/notify-me emails for landing pages. I was about to build a YAML file or use SQLite3 to store emails via form, but this is so much more convenient.

Ah man, this is perfect! finally a way to get simple servers running with very little hassle.

One major feature I would like is the ability to specify cells for I/O. Eg in some sort of "my api" console I could say "/custom_api, {stuff:A5, things:A6}, B2". Then requests to GET /custom_api would plug in the key-value of "stuff" into A5 and "things" into A6 and then respond with whatever is in B2.

Once more useful features are up, the ability to disable parts of the API (eg dump the whole spreadsheet) would be necessary.

Obviously you wouldn't build a large app with this, but I could see many traditional businesses using it for more than just prototyping.

Thanks for the ideas! I'll consider them.

You can do that with SpreadServe[1] now. Instead of Google docs it uses real Excel .xls spreadsheets and supports VBA and XLL addins. It turns your spreadsheets into async scriptable servers. There are Python & Java APIs.

[1] http://spreadserve.com

Is this open source anywhere or just closed beta?

Some OSS here: https://github.com/SpreadServe Sign up for the beta here: https://groups.google.com/forum/#!aboutgroup/spreadserve AWS hosted instance: http://54.148.111.119:8888 Docs: http://spreadserve.readthedocs.org/en/latest/

You can do that also with http://calcfusion.com ( disclaimer I work there ).

Neat. Instantly converted my BTC price tracking spreadsheet [1] into an API [2]

1) https://docs.google.com/spreadsheets/d/1iUVXmC04KIU5K1Osb_4H...

2) http://sheetsu.com/apis/2aab25c7

Note: if you changed the first row to show the type of data that will follow, that API will be tons nicer to use.

Google Spreadsheets already have a native REST API. I use it to feed the charts in http://pittsburghtoday.org/ The key to getting good performance is to not put too much data into the sheets. Low thousands of rows is not a problem.

Was this inspired by a Danielle Morrill tweet by any chance?

Yes, it was. I've similar problem like she had. I started looking for some good solution. Didn't found anything, so I wrote it myself.

I twitted her, but no response from her. If you have contact with her and she still needs it, please let her now about Sheetsu.

Really great start! I could see this being very useful.

If you were to extend the functionality to include full CRUD on entities, I think I would start using it immediately on some proof of concept work.

For instance, being able to GET, PUT/PATCH, and DELETE by id (or per row) would be awesome.

Thanks again for the nice MVP work!

Thanks! I'm planning to implement PUT and DELETE soon.

I've used sheetrock (https://github.com/chriszarate/sheetrock) with good success in the past. But this definitely seems more refined.

Another discussion 6 months ago triggered by a Blockspring (SaaS tool) blogpost:

https://news.ycombinator.com/item?id=9258774

Nice way to bypass zapier.com for some simple stuff because they can get expensive quickly. I just started using it to store a log of supersaas.com appointments in a Google spreadsheet, works great.

I see that you request access to 'view and manage' the spreadsheets and files in my Google Drive? Is this for each spreadsheet/file or just the one I've linked to the in URL?

Right now it's for all. I'm actually fixing it right now, to be access only for the one specific file.

Perfect! That was the only thing holding me back from getting started. Nice work, BTW!

This is great! I want to build an app that can insert transactions on the go into my existing budgeting spreadsheet, and this really takes away a lot of the dirty work.

So I threw this into a website to try and load some data into a table, and get: XMLHttpRequest cannot load http://sheetsu.com/apis/d54d6315?sort=Description&order=asc. No 'Access-Control-Allow-Origin' header is present on the requested resource. How do I handle that?

I ran into the same thing. Sheetsu apparently doesn't have CORS (http://www.html5rocks.com/en/tutorials/cors/) enabled so it won't work from client-side JS right now.

Already added to the to do list. Thanks for noticing it.

An alternative is also http://calcfusion.com/ ( disclaimer, I work there and we just launched ). It uses xls spreadsheets and is more oriented towards business rules and computations. There are some nifty features like audit and versioning, automatic html form building etc...

Or you could roll your own with this npm package I wrote which turns a google sheet into static JSON:

https://github.com/techjacker/google-docs-cms

On my server I have a cron script running which pulls the new JSON if there have been any changes.

Sheetsu is pronounced the same as "Chie dessus" in French, meaning "crap on [it|you|...]" :)

Hahaha :) It was supposed to sound like Shih Tzu, breed of dog - http://jp12.r0tt.com/l_d67efae0-58bd-11e2-baa5-afb676b00012....

Will you maintain relationship between sheets? For many API use-cases, I'd expect my data to be relational.

It's great idea! I'll think about it. You want use each sheet as a separate table (like in DB), right?

Exactly.

I didn't thought about it yet, but right now when you put =SUM(A1:A10) into cell, and GET (or POST) it via API, you will get the result (sum of the column 1-10). So, it'll be working with sheets the same way.

ex. when you put =Sheet2!A1 and get that via API, you will get the value of A1 cell in Sheet2.

Slightly unrelated but - isn't there a super cost-effective way of quickly whipping up a CRUD API with minimal configuration? Often I implement fairly backends in Java or Go that could might as well have been "configured" from the beginning, instead of implemented each time.

Parse or Firebase come to my mind... both have very generous starter plans.

https://www.firebase.com/docs/rest/quickstart.html

https://www.parse.com/docs/rest/guide

If you're into node, you might want to check out json-server. A single json file with one sample entity is the basis of a full API. Check it out at https://github.com/typicode/json-server

Or FakeRest for that matter. Why would you set up a server when everything you need is a browser ?

https://github.com/marmelab/FakeRest

Whoa, looks neat and honestly sounds too good to be true.

>Intercept XMLHttpRequest to fake a REST server based on JSON data. Use it on top of Sinon.js to test JavaScript REST clients on the browser side (e.g. single page apps) without a server.

I'll check it out this weekend, but man... thanks for sharing!

I've got submited to Product Hunt[1].

[1] http://www.producthunt.com/tech/sheetsu

Is there a way to get data from each sheet in a workbook? I'd love to use this, it is so simple. But it only seems to get the first sheet in my workbook.

Yes, it could be done. I'll implement that feature.

Awesome! FWIW, my needs don't require any sort of "relational" concept as some comments have suggested, I just need to be able to do all existing operations on each sheet in the workbook. I guess being able to get a list of the sheets in the workbook might be useful, as well, or get a sheet by name, but for my use case I already know the name.

Awesome API, but one question. How do you prevent other people from doing a GET once they know your sheetsu API url?

How is this any different than just getting the json feed for the sheet? I find it much more flexible.

You can POST your data. PUT and DELETE soon.

That'll be great. Is CORS support on the roadmap too?

Yes, it's. - PUT/DELETE - CORS - more sheets in the same workbook - access for only one file in the Google Drive - private APIs

Cool. Is there any API rate limit?

Right now, it's Google Drive Quota. I'm testing out the idea, etc. If it goes well I'll probably cut rate limits per user or per API.

But google throttles requests to popular docs right? I mean, the "Wow, this file seems very popular" thingy... Is there any way to calculate it? Maybe you could pass JMeter [0] or bench-rest [1] so we can have some numbers for orientation?

Anyways, it's really handy for simple and non traffic-intensive projects. Well done!

[0] http://jmeter.apache.org/

[1] https://github.com/jeffbski/bench-rest

Paste Google Spreadsheet url and get a link to an RESTful API. Read (GET) and save (POST).

What is it that people think "an API" means, exactly?

Why not speak normal English and say "access a Google Spreadsheet via an API", or something?

wow, it requires to access all my googledrive data and the spreadsheets.

This'll be fixed soon!

Every year, twice a year someone posts this solution. We get it. You hack.

So? Out of the thousands of people that frequent this site, a tiny percentage of them make this, and it's cool for people that have never seen it.

Also, relevant xkcd. https://xkcd.com/1053/

no. cause then eeeeeeeveryone goes "oh cool I've done the same thing." so the only take away is that people don't learn from the past. sad.

That's an extremely dismissive comment. He did something, and is telling people. Good for him.

http://carl.flax.ie/dothingstellpeople.html

API doesn't look very REST to me. It is basically just using HTTP as a JSON transport. The JSON contains a "status" code which appears to loosely follow the HTTP status code numbering. But if that was the intent then why not just use HTTP status codes like a true REST endpoint?

Otherwise, a neat idea. Needs a Office 365 connector too.

It's an MVP right now. Yep, will introduce those fixes this week! Thanks for kind words.

Microsoft (specifically Power BI) is pretty keen on consuming OData so exposing a Google spreadsheet as OData might be interesting.

Salesforce loves to consume OData too. I'd be very interested in a sheet->OData connector.

true REST != JSON :D

yes json it's convenient. no json isn't stateful, unless you transmit operations and allow API discovery, not just data.

Reply on news.ycombinator.com