Google Cloud Datastore | Hacker News Reader

Google Cloud Datastore

developers.google.com

41 pointsdeno8 comments

We are using the Google Cloud Datastore for quite some time at Blossom.io (through App Engine) and it is an amazing piece of technology.

Since scaling the data-tier is the most challenging part of building a high traffic application (especially when it comes to scaling writes) the Google Cloud Datastore (& App Engine) imho is among the very few services that can actually claim to solve this problem whereas other platforms just try to market theirs this way – "go cloud and then you can scale automatically (if you know what you do)".

I'm really glad the Datastore is now also available outside of App Engine.

It would be really cool to see a secure JavaScript API so you can query the datastore from a client-side JavaScript page like you can with Firebase.

This is not new. But how good is it anyways?

It’s highly‐available NoSQL database (Megastore) with ACID transactions based on entity‐groups[1]. No other cloud provider has anything even remotely comparable. And it’s now available outside of the App Engine framework.

I was hoping they’d make App Engine to actually run on GCE, but this is fine for now.

I’m a little bit worried about this though:

> You should be aware that Cloud Datastore has a serving component that runs on Google App Engine, so there will be instance hour costs.

What I think is going on is they are either worried about about DDoSing their internal APIs or they’re not willing to opensource the part of the client that runs on App Engine instances behind SDK. I think there’s a tech talk coming up on Cloud Datastore that may shed some light.

[1] http://www.cidrdb.org/cidr2011/Papers/CIDR11_Paper32.pdf

I mean I remember reading about this a couple of days ago. I guess "outside the cloud" is what I missed.

Yes, high availability with ACID and 'eventual consistency' on reads. I never understood why an eventually consistent datastore could qualify as ACID because in the real world if you need ACID, then eventual consistency is never good enough.

It's good to see Google roll this service out separately from GAE but it's not as good as DynamoDB or data services from Parse and others.

You only have eventual consistency for queries outside of entity‐groups. For ancestor queries you get strong consistency.

https://developers.google.com/appengine/docs/python/datastor...

Imagine a comment system. You’d fire one global query with eventual consistency guarantees for all comments in a thread and another one in parallel for only comments made by the signed in user. Merging the two gives you the best of both – great scalability and consistency where it matters.

> it's not as good as DynamoDB

DynamoDB’s transaction support is limited to incrementing counters and optimistic concurrency on a single item. Compared to Datastore, it looks like a toy.

My entire business is built on top of the datastore running on GAE. I've heard stories from googleers about loosing an entire datacenter and nobody on the outside noticed. Even with the limitations of developing on top of the datastore, there is some really amazing technology around it that sure makes it worth using. Exposing the datastore to GCE is a step in the right direction.

Reply on news.ycombinator.com