Show HN: EdgeDB – Next generation database
edgedb.com
edgedb.com
1) New database product that's an alpha. My first thought is "will be interesting to follow for 5 years to see if they make it". database choice is possibly the single biggest decision I make in my role; I don't make it lightly.
2) It looks like it's trying to bridge the NoSQL/SQL gap -- more consistent for SQL fans, more flexible for NoSQL fans. (My judgement: "serving 2 masters is not a great signal for me to choose this DB" but I'm a SQL curmudgeon.)
3) GraphQL is really neat out of the box. Some neat features overall.
And, finally, I'm still not really sure what situation could possibly arise that would make me choose this DB, mostly because that choice isn't something I'd select a <5 year old solution for.
I do like that it's based on Postgres (Citus taught me how powerful the "build on pg" model can be). I really like the graphql and schema introspection features. I don't think I would put this on my vendor/technology comparison matrix.
Hope this is helpful, please don't read too much into the negative stuff -- if I were choosing any other technology pillar besides a db, I wouldn't be nearly as strict. It's an uphill climb for your team, but I wish you the best of luck!
I agree, people do choose a database based on its track record, and this is where EdgeDB will have to prove itself. Overall, I think that the decision to build on PostgreSQL is a great competitive advantage over other new databases: we can confidently say that the data will be safe with EdgeDB. And we can actually spend more time improving the ergonomics of the data model and the query language.
One thing I've rarely seen mentioned, but I believe is a force behind MySQL's success: metadata manipulation. DESCRIBE TABLE, SHOW CREATE TABLE,... stuff ends up being a lot more useful than I would think as an outsider. Going from MySQL to pgsql was a steep learning curve because the commands to display table listings (\dt; i think) felt very arcane. Also, being able to rename, add, remove columns. Finally, can I create a table with the same schema (data optional) as another table? I do that a lot.
I don't want to touch a database product until it has a proven production use for several years.
Creating a database is hard, very hard, as history has shown us recently.
That it sits on top of Postgres is a great start though.
- transactions - proper paging - indexes - query engine - and more ...
However, if they use postgresql as a base.. it significantly improves the stability and feature completeness. Especially if it is only a different frontend (query language and schema).
My only concern is that there is nothing about the backend that suggests that this will be scalable beyond tiny data sets. Even on single machines, there is a performance cliff that arrives very quickly when doing these kinds of data model traversals unless you are using something exotic under the hood.
Thank you!
> Even on single machines, there is a performance cliff that arrives very quickly when doing these kinds of data model traversals unless you are using something exotic under the hood.
PostgreSQL is used under the hood. We compile EdgeQL to an efficient SQL behind the scenes, similarly to how you compile a high-level language to lower-level language. All our fancy data types fully utilize the relational model, so what's achievable with scaling Postgres will be achievable with EdgeDB. I know quite a few massive deployments of Postgres that work just fine, there's simply no reason why EdgeDB would be different.
An occasional post tracking the status of that upstreaming effort would be very welcome.
We've already found a few bugs in Postgres and upstreamed fixes for them. There is a patch or two from us in their review pipeline right now.
May I assume that the slight modifications you were required to make to postgres in order to support your features weren't just bug-fixes?
Yes. Some of them, like strict parsing of timezone-aware vs naïve datetimes can be upstreamed (as a configurable option) if the Postgres community accepts it. It's in our interest to upstream things like that :) We'll see how it goes.
btw, do you implement any edgeql-specific optimizations? I know that the db shares most of the code with psql, and its optimizer is okay. But being closer to the math foundation opens up lots of opportunities..!
Not on the planner level. Most of EdgeQL-specific low-level stuff is currently built as an extension with helpers. Our strategy is to upstream as much as possible, including any possible improvements to the planner that benefit EdgeDB.
So how is a missing value in a column represented? A column value is a scalar value, not a "set". As much as I accept that NULL values are a pain to deal with, I don't see a way to represent the absence of information in another way. You can't store an "empty set" in a column.
A NULL _value_ in SQL is something completely different than a NULL reference or pointer in a programming language. Although both have the same name the underlying concept is completely different.
> Functions in SQL can return NULL to signal an error condition.
That's plain wrong. functions in SQL signal errors by throwing an exception
Explained in this thread: https://news.ycombinator.com/item?id=19641060
> > Functions in SQL can return NULL to signal an error condition.
> That's plain wrong. functions in SQL signal errors by throwing an exception
Well, just as an example: take a look at the 'to_number()' function. It will happily return NULL if the format string is an empty string. An empty format is an error condition (it's not documented to return NULL, btw, see [1]) at least in some contexts, so the returned NULL breaking the query can be very unintuitive.
So yes, throwing an exception is what happens in 99% of situations, until you hit a weird case where a NULL is returned unexpectedly for you. And I'm not saying that returning NULL is bad for 'to_number()'; in SQL that's OK. It's just not how we approach things in EdgeQL.
[1] https://www.postgresql.org/docs/11/functions-formatting.html
Btw I suggest coming up with a better name than EdgeQL. Maybe just EQL?
EdgeQL as a language is so intuitive and advanced only because it operates on a very specific data model that can be compiled to a relational model, but isn't really compatible with it. So naturally, EdgeDB wants to be in absolute control of the underlying Postgres backend, which goes beyond of what's acceptable for a Postgres extension.
> Btw I suggest coming up with a better name than EdgeQL. Maybe just EQL?
There are other languages called EQL (at least that's the impression I got when I last googled it), whereas EdgeQL is unique :)
This is a blog post about EdgeDB 1.0 Alpha 1. It's a new object-relational database, focused on type safety, usability, and developer ergonomics. We hope you will try it out!
Their is one point I'd like to address, though. It's still a real pain to write queries from within other programming languages. You either write them within strings, which sucks, because you lose all support from the editor (highlighting, auto-formatting, auto-suggestions, etc.), or you use an ORM, which I think is not a good solution to the problem. If you want to boost "developer ergonomics", make editor plugins for EdgeQL first class citizens. My 2 cents. :)
> You either write them within strings, which sucks, because you lose all support from the editor (highlighting, auto-formatting, auto-suggestions, etc.), or you use an ORM, which I think is not a good solution to the problem. If you want to boost "developer ergonomics", make editor plugins for EdgeQL first class citizens. My 2 cents. :)
Yes, that's exactly what we are going to do. We'll obviously add a query builder to edgedb-python and other language bindings. We also plan to reflect the DB schema to your language of choice so that the query builder is fully typed. Then IDEs will have no problem with auto-completing your code.
Another thing we're considering is implementing the Language Server Protocol for EdgeDB.
I'm sure you would need to cut off many features but I would start small and release it as a Postgresql extension. Even better, you can sit in front of Postgresql and develop your own client libraries for DDL and queries. That way, you could support RDS and Cloud SQL so companies can confidently try out EdgeDB. Once you prove that the new query language works, you can extend and build a new database from Postgresql.
What is the minimal system requirements? And what about partitioning, replication, event log, etc.
Is there a native way hook to graphql resolvers? I really like it either way, there some tools for composing and stitching schemas.
plans for hosted and managed solution?
Did you consider "pageInfo" type, maybe it is just me, but it proved very useful on frontend.
instead of `Movies: [Movie]`
Movies {
pageInfo: PageInfo # { hasNextPage, endCursor }
nodes: [Movie]
}Minimal requirements are similar to that of PostgreSQL. EdgeDB should run comfortably on an average server.
> And what about partitioning, replication, event log, etc.
Tooling for that will be coming in the next few alpha releases.
> Is there a native way hook to graphql resolvers? I really like it either way, there some tools for composing and stitching schemas.
Not right now. Although I think it should be possible to do what you want via one of the existing GraphQL proxies, like Apollo.
> plans for hosted and managed solution?
Yes, but no ETA right now. This is one of our highest priorities.
> Did you consider "pageInfo" type, maybe it is just me, but it proved very useful on frontend.
We can add that. One of the first-priority things on our list w.r.t. GraphQL though is to implement Database Views, which will allow selectively exposing your schema (and controlling how it is exposed).
Could you clarify why the benchmarks show lower performance than vanilla Postgres? Is it because of less efficient generated SQL? (I am not a database engineer and probably missing something obvious)
Lower performance in benchmarks is explained by an I/O bottleneck in the EdgeDB server, the queries are not less efficient. There will be performance improvements in subsequent releases.
Still nothing so far.
EdgeQL is a fully featured query language, with support for subqueries, aggregation, transactions, rich datatypes like tuples/named tuples, arrays, JSON, etc.
The Data Model is object-relational, with support for multiple inheritance of object types and links between them. GraphQL really shines when it's used to query EdgeDB, it just feels natural.
That said, GraphQL can only do relatively simple queries, like "fetch me a hierarchy of objects with this shape with some basic filters". EdgeQL, on the other hand, allows you to build queries as advanced as what you can do with SQL.
Finally, there's only one schema in EdgeDB. You don't need to maintain a separate GraphQL schema, like you need to with hasura and other GraphQL solutions.
Are arbitray-depth graph queries a design goal, or are the nested results just for optimizing network traffic?
On the contrary, we optimized it for the kind of applications that are usually built with relational or document databases. Those scenarios frequently involve a relatively complex schema with queries that fetch hierarchies of objects 2-3 levels deep. Doing that efficiently in SQL or NoSQL isn't as simple as it sounds (and the blog post makes a point about that).
That said, I expect EdgeDB to perform on par with neo4j on moderately deep object hierarchies. We also plan to add support for recursive queries at some point, although it's not a priority right now.
> queries that fetch hierarchies of objects 2-3 levels deep [...] efficiently in SQL or NoSQL isn't as simple as it sounds
It doesn't sound simple and it isn't simple. :-)
``` issues_query = Issue.objects.filter(location__contained=bounding_box) if all_of: for tag in all_of: issues_query = issues_query.filter(issuetag__tag=tag) if any_of: issues_query = issues_query.filter(issuetag__tag__in=any_of) ```
I really like some body is finally doing something to "SQL" in terms of the language itself. I hope this catches up and other will consider doing something similar.
I expected EdgeDB to be a database suitable for edge computing, namely able to run globally, near the edge compute nodes, and replicate data with eventual consistency.
Now you can't tell if it's empty or just not set.
Now I have to track another thing just to tell me if I have touched a thing.
Please don't tell me this is some kind of nulls were a mistake cargo culting?
Nulls are just option types with good usability. Not the languages fault if you decide to put a tripple meaning on it.
Those are the same things in EdgeDB. You can have a "required link", in which case a relationship between two objects is required, and therefore the link cannot ever point to an empty set. Alternatively you can also declare your link as "optional", in which case you have it pointing to an empty set (essentially to "nothing"). You can coalesce that empty set with a non-empty set, if you need.
That is similar to SQL; the key differences here are:
* The boolean logic in EdgeQL is two-valued, it's "true and false", not "true, false, and NULL". Some people find it hard to guess what "true OR NULL" returns in SQL, because unless you are a DBA, this isn't something you deal with on a daily basis.
* Functions/operators in EdgeQL never return empty set to indicate an error condition. This is one of my personal pet peeves with SQL; in some situations it's hard to understand why a big and complex query yields slightly wrong results sometimes.
> Now I have to track another thing just to tell me if I have touched a thing.
I suggest you to try writing EdgeQL queries. The experience is usually quite the opposite, and I hope you'll end up liking it!
> Please don't tell me this is some kind of nulls were a mistake cargo culting?
While I do think that NULL is a hard to deal issue in many languages and I admire languages like Rust without NULL, EdgeQL is slightly different. As you noticed, we do have a concept of empty set, which in some ways is similar to NULL in SQL (with the caveats I explained above).
Why would you admire this? Dropping .unwrap().wrap() on all the things is not any better than doing the same thing on nullable types.
"duh you're meant to handle them straight away so its not an issue" - no shit? Do you think maybe the same could be said about null values?
Removing optionality from a type system is a mistake, it just leads to more gibberish to handle undefined/notset.
> EdgeQL is slightly different. As you noticed, we do have a concept of empty set, which in some ways is similar to NULL in SQL (with the caveats I explained above).
Yes, it is slightly different, no it is not similar to null. It is what you get when you don't have null and don't have options. i.e. no sentinel, just a maybe.
Very simple, you leave it out! Then you can query for this.
- You can define a `bool` property as "optional" and then it can have an empty set as its value.
- The 'exists' operator coverts sets into bools; you can use it with the 'if..else' operator to handle empty sets where needed.
- You can coalesce (the "??" operator) your potentially empty property with a non-empty set, say a "{false}".
- An empty set isn't some magical value like NULL in SQL. Functions and operators in EdgeQL are strictly defined if they accept sets as their arguments, or not. In the case of logical operators -- they don't; they are defined as element-wise operators, therefore no three-valued boolean logic. You learn the rules once and they apply to all functions and operators in all contexts.
This documentation page (and subsequent sections) explains how exactly EdgeQL is defined: https://edgedb.com/docs/edgeql/overview.
Here are some options:
1) count(NULL) = 0, so apparently NULL is just like an empty set at least some of the time. This means that some of the code will treat NULLs as empty sets and other code will not, leaving the burden on the programmer to keep in mind these implicit differences.
2) count(NULL) = 1 because NULL is a value, albeit a sentinel value. This can lead to tricky problems where a count() suggests that there's some data, while, in fact, there is none.
3) count(NULL) = NULL. If the idea behind this option is that sentinel value cannot be operated on, then this is pretty much like throwing an error at every NULL, which, in turn, will result in the necessity to guard many expressions with some error (NULL) handling code.
One of the things to note is that an empty set has rather unambiguous semantics, while a NULL presents options each of which can be justified depending on how a particular person thinks about this special value. The line between "a value has not yet been assigned", "no value has been assigned" and "a value has been unassigned" is very blurry. On the other hand, the line between "there is no value" and "there is a value" is pretty clear.
Null is a word, it has a meaning. In programming that meaning is the absence of a value.
For ergonomic reasons dynamic languages will cast it to 0 or false in some situations. But that doesn't change its meaning and it doesn't mean it has multiple meanings.
That would be like complaining about how most languages treat 1 == true and 0 == false.
> So then the question is this: what should count(NULL) be?
No, the question isn't that.
The question is what is null. It is null.
Count(null) does not change what null is. Null is still null.
If count wants to treat null like a 0 for ergonomics it can do that. If it wants to treat it like an empty set it can do that too.
If count wants to treat null as U countable and throw an error it can do that as well.
If it wants to treat it as a null operation and return null it can do that too.
Personally, I find 2, and 3 fucking useless so if it was my language, I would define a countable interface for null and set null = 0. Because there are no usecases where you want that to error at runtime.