wsh91

Hi there! One thing you might consider is using a function to replicate your Firestore data to BigQuery, depending on the value the latter would add. (Very fast OLAP; I'm a big fan.)

(Cloud Firestore eng. here.)

Hi there. :) I'm an engineer on the Cloud Firestore team. I can't speak to if or when we'll be able to remove this restriction, but please know that the folks working on this (like me) are very aware of what a pain in the butt this is. I'm sorry that I can't give you a more satisfying answer, but I hope we'll be able to change this soon. Thanks for being a customer!

Cloud Spanner is also relational. :)

(Fellow GCPer.)

An export is one component of a backup solution. :) Scheduling is another (https://firebase.google.com/docs/firestore/solutions/schedul...). (Checking restores is critical, too; that's also feasible [managed import into a new cloud project] but requires a bit more legwork.)

(I work on Cloud Firestore.)

Is there anything we could be doing to serve you better?

a) Cloud Firestore supports transactions across the entire database. You can learn more about them here: https://cloud.google.com/firestore/docs/manage-data/transact....

b) Given that the primary use case for namespaces was/is multitenancy, it's not clear to me why you'd want to transact across them. Nevertheless, you can. What's leading you to draw this conclusion?

Cloud Firestore (the next generation of Cloud Datastore) removes those limitations. https://cloud.google.com/firestore/docs/manage-data/transact...

It also supports the Cloud Datastore API.

(I work on it!)

Hey there--I'm an engineer on the Cloud Datastore team. I'd love to know more about what your needs are if you're willing to share.

This (fantastic) paper is cited in the Google Megastore paper [1] and we actually use the term "entity group" in our product [2] indirectly because of it. Pat Helland is always worth reading; thanks to the submitter for the reminder.

[1] https://static.googleusercontent.com/media/research.google.c...

[2] https://cloud.google.com/datastore/docs/articles/balancing-s...

Have a look at https://cloud.google.com/compute/docs/load-balancing/ipv6.

(Disclaimer: I work on GCP, albeit not on networking stuff.)

Only because this question has been asked multiple times at TGIF (regular all-hands meeting).

That's not true. Anyone has access to 20% time, the question is just whether or not it makes sense for individual people.

Source: I work here and take advantage of 20% time.

Heh. :)

One technicality: DynamoDB's pricing is based on throughput, as you know, but it's provisioned throughput. (You manage your capacity unit allocations yourself, at least that was the case a little while ago.) You're charged money for how much you provision regardless of whether or not you actually consume the capacity units. Our pricing model (like Fauna's) only takes ops and storage into consideration.

I can't speak to the SDK parts, but take a look at Firebase's suite more broadly. :) Viz a viz Cognito, check out Authentication and the Realtime Database in particular.

Firebase's Realtime Database does this, too.

https://firebase.google.com/pricing/

(Disclosure: I work on Cloud Datastore, which is another Google Cloud product; it, too, does pricing based on ops and storage rather than provisioning, as my colleagues note.)

As you noted earlier, Megastore has a schema and we don't, so we have our own index implementation, yes. :)

Perhaps I've misunderstood you, but I'm pretty sure cross-partition (I assume you mean cross-entity-group in our terms) transactions are in fact consistent (not totally sure what you mean by transactions being consistent, per se; if you're talking about serializability, at least, we are). Explicitly (from [1]):

  Queries that participate in a transaction are always strongly consistent.

And the consistent write throughput to which you refer means sustainable write throughput per entity group. We can burst much higher.

It would be much easier to assess the relative consistency models of our products if FaunaDB had documentation with respect to its claims. We have a litany of pages about ours (e.g. [2] and [3]).

[1] https://cloud.google.com/datastore/docs/concepts/structuring...

[2] https://cloud.google.com/appengine/articles/transaction_isol...

[3] https://cloud.google.com/datastore/docs/concepts/transaction...

Again, neat stuff, but that's not what this link claims:

  * The first serverless database
  * The first active-active multi-cloud database
  * The first strongly-consistent multi-region database available to the public

None of these are firsts. I don't know if our (GCP) services are themselves the first of their kind (it's an ambitious claim and, as an engineer, I try to be careful about those), but Datastore meets at least two of those three and predates FaunaDB by several years.

I'm familiar with both; what disappoints me is the claim of novelty here with respect to autoscaling. That's just not true. To quote you:

"A serverless system must scale dynamically per request. Current popular cloud databases do not support this level of elasticity—you have to pay for capacity you don’t use. Additionally, they often lack support for joins, indexes, authentication, and other capabilities necessary to build a rich application."

That first criterion we absolutely meet, today. Cloud Datastore has been doing that for eight years now. We don't have joins, but we do have indexes, auth, multi-region replication and a whole lot more.

:)

Here's a paper with which you might already be familiar, but it's one of the citations for the Megastore paper: http://adrianmarriott.net/logosroot/papers/LifeBeyondTxns.pd.... You'll probably enjoy it (if you haven't already!).

We also have transactions [1], compound indexes [2] and multi-region replication [3]. I'm not sure what defines a "full-blown" database, but that much is true.

[1] https://cloud.google.com/datastore/docs/concepts/transaction...

[2] https://cloud.google.com/datastore/docs/concepts/indexes

[3] https://cloud.google.com/docs/geography-and-regions#multi-re...

Datastore runs on top of Megastore [1]. You can find out more about our data model here [2], but it's definitely not limited to key-value data.

Our end users don't have to think much about our storage system, though, if we're doing our jobs right. :)

[1] https://cloud.google.com/datastore/docs/articles/balancing-s... [2] https://cloud.google.com/datastore/docs/concepts/entities

(Disclosure: I work on Google's Cloud Datastore.)

This looks super neat, and I can't wait to learn more about it, but just for the record: I'm pretty sure this isn't the first serverless cloud database. Both Firebase's Realtime Database and Cloud Datastore (which powers Snapchat and Pokemon Go) are serverless; you pay only for your ops and storage. They've been publicly available for several years.

GAE gives at least a year after deprecation before turning off a feature. https://cloud.google.com/appengine/docs/deprecations/

(I work on Google Cloud and on Datastore, which is built into App Engine. We bend over backwards to support existing code.)

Google Cloud Storage (as mentioned) for storage, Google App Engine for PaaS. :)

(I work on Cloud, specifically Datastore.)

The question didn't even mention GPUs, and there's no reason that what you mention can't change in the future. It was a pointer.

There are discounts if your instances can be pre-empted: https://cloud.google.com/preemptible-vms/. The pricing isn't variable like EC2 spot instances, however.

Cloud Datastore has a perpetual free tier, too: https://cloud.google.com/datastore/ :)

(I work on it.)

Really sorry to see this; your candor is refreshing, and I've been interested in the optimization work you've been doing for some time. Best of luck with what comes next.

Spanner's p90s are, for at least one user, consistently lower than 50 ms. https://quizlet.com/blog/quizlet-cloud-spanner

(Disclaimer: I work on Google's cloud.)