What I want is a new database that defines tables in terms of rich structures like proto, like Spanner.
If that doesn't exist in a couple years, maybe I'll try to build one.
If that doesn't exist in a couple years, maybe I'll try to build one.
Do you mean like https://www.edgedb.com/? Or something more... rich?
It's very interesting, but one reason I want it to be proto is so you can re-use that for bindings with your applications.
Having and independent schema file being one to one with the proto files is not quite as nice, but then you still can get bindings in each language to the proto.
The goal is to avoid defining the table schema in each language, like ORMs, where things can be out of date.
There may even be some benefits to this, as you might have some lifecycle hooks around migrations to keep the proto up to date with the current schema. You could do this with just protoc plugins to hide fields in the language bindings or you could do it turing the edgedb schema to proto translation.
Not sure you need a completely new db though; with protobuf’s native support for plugins, perhaps the missing piece you’re looking for is a protoc plugin that generates the table definitions for your db of choice? Could work well for databases with declarative schemas..
Don’t want to discourage you from creating a new database, but writing a proto plugin is arguably a quicker undertaking!
What I'm unsure of is how well proto can be modeled in various existing relational databases and what the performance and management consequences might be.
If I have a message defining a table and it has a field which is another message, how well can Postgres Structs represent that submessage? What are the performance implications? I'm not terribly familiar with the query semantics for Poatgres structs. How do migrations and backfills work for struct fields and nested structs?
These considerations may or may not justify a new database.
But like a sibling comment said here, if you want the database implementation to dictate your models you can probably generate protos from something like EdgeDB. But then the db implementation is in charge, which seem a bit backwards.
If you’ve used graphql you’re presented with a similar challenge but on the other end of the spectrum - the graphql schema that needs to map to application models.
So one of the main considerations is to figure out how and where you want to define your source models. Is it the view layer, the model layer or the database layer.
Tldr; yes you’re right, it depends!