EdgeDB: EdgeQL | Hacker News Reader

EdgeDB: EdgeQL

website-atgsmhega-edgedb.vercel.app

124 pointstodsacerdoti89 comments

Hah, I'm not sure how our dev staging URL ended up on HN :)

Here's a direct link to the documentation page about EdgeQL: https://www.edgedb.com/docs/edgeql/index.

EdgeQL, the query language of EdgeDB, is a very interesting piece of tech. It's a new, strictly typed query language, that aims to surpass SQL in querying power. Functional in its nature it was designed to be composable and easy to learn. Happy to answer any kind of questions.

EdgeDB itself is almost ready for its 1.0 release, stay tuned. The GH link: https://github.com/edgedb/edgedb/

// Disclaimer: I'm a co-founder & CEO.

Wow, EdgeQL looks amazing! I've spent a good amount of time looking into SQL alternatives and building my own, and this is the only one that feels like "I must try this!" (Too bad my current project has no need for a DB; now I have a dilemma...)

> You can run EdgeDB as a query engine on top of an externally hosted Postgres instance or let EdgeDB manage Postgres for you under the hood.

This is excellent. Allows me to use EdgeDB for the majority of my application, but still treat it as a "plain old Postgres DB" when needed.

I found EdgeDB today via the article "Against SQL" (which triggered so many pain points), and now rewriting one of the projects to use it. I know it's not production ready, but it looks too good not to try.

Let us know how it goes!

Thanks for the kind words!

Also feel free to play with EdgeDB in your browser with our interactive EdgeQL tutorial: https://www.edgedb.com/tutorial

I'm a SQL newbie so maybe I lack context for this decision, but why did you decide to keep the "SELECT" keyword? Going through your tutorial, SELECT seems redundant and could just be dropped without any loss of information. In that way look a lot more like most non-SQL languages. Why do I need to SELECT "string" instead of just "string"?

Off the top of my head: EdgeQL grammar is relatively complex and we need SELECT to disambiguate in many corner cases.

I'm trying to understand how this tutorial works, I noticed it doesn't make any network requests when run one of the examples, but then if I edit the code in an example it does make a request. Is this the magic of next.js and server-side components at work, and do you have a real edgedb running on the server side that is used to pre-render the page and also handle updates? I would love to hear more about how this setup works and even look at the code if it's available.

You guessed everything right: we run an instance to prerender and cache at the build time and at runtime we query when the query has been changed

If there is one piece of techno I have my eyes on since some time, and am the most exited about, it’s EdgeDB. You guys have nothing to prove anymore. I’m using your work on a daily basis: asynpcg, uvloop, black and … python !

When I came across the EdgeDB project, I thought: “damn it’s too good on the paper to be true”. And when I watched the 6th video of “import asyncio” from Łukasz on your YouTube channel, it blew my mind the kind of queries one could achieve, and how flexible was the schema definition with the abstract types.

Although I’ll have to be a bit more patient to push this techno at work, I’m even orienting my personal projects to be a fit for edgedb usage.

Please keep up with the awesome work !

My top features on the roadmap for 1.x:

- LSP

- Query Builders and Schema Reflection

- Fine-grained Data Access Control Rules

Thank you :)

Do you have any simple mechanism for bulk import, either from CSV or JSON files, or existing Postgres? Googled a bit and didn't see how to bulk load an existing dataset.

There aren't any special "bulk" interfaces at the moment, you can load data via a small program using our language bindings (INSERT in a loop). We'll add support for batched `executemany()` soon to cater to very large loads specifically. A similar optimization in asyncpg [1] led to a 10x improvement in throughput (~100Krec/sec per connection), and we expect similar performance from EdgeDB once the protocol and the bindings are updated.

Loading arbitrary JSON by introspecting the data, and auto-generating the schema and the necessary DML statements is fairly straightforward too. I have a rough Python proof-of-concept, will need to find time to finish and publish it.

[1] https://github.com/MagicStack/asyncpg/pull/295

Really nice 1st1 - congrats, all your resources look so clean.

Thanks! :)

> that aims to surpass SQL in querying power.

What makes your approach likely to succeed versus all previous attempts to supplant SQL in the last 40 years?

We've spent years in R&D mode working on the data model, the query language, APIs and architecture to make sure that everything clicks together to create one vertically integrated, cohesive, and language agnostic solution. Everything in EdgeDB is optimized for DX (while not sacrificing perf or type safety), and as far as I know there's no other database company out there which does what we do. So fingers crossed, our attempt at evolving relational databases will be successful.

Everything you said sounds like Britton-Lee's database. What is new?

I'm not intimately familiar with that particular piece of history, unfortunately (I'll read more about it for sure). There were many examples in the past of great tech (like smalltalk or network dbs) that didn't exactly succeed for many reasons, sometimes because it was created way ahead of its time. We build EdgeDB to address today's needs: nested hierarchical queries, integrated schema migrations, performance, high quality language drivers etc. There is some intersection with what was done in the past for sure, but if you're interested in databases I encourage you to take a look at our website, the blog, and the docs. I'm sure you'll find new things.

Just discovering EdgeDB. How does it handle graph queries (specifically, traversing joins across > 3 tables under the hood)?

It handles them just fine. Any single EdgeQL query always compiles to just one SQL query under the hood. We wrap nested hierarchy levels subqueries in array_agg and our binary protocol is designed in such a way that unpacking the results is very fast.

Question: why do you claim EdgeDB is a "relational" database when you can't actually do relational algebra with it? How do you join?

Materialized edges (i.e. what you call "links") are not part of any relational model, it's a part of the graph model. You seem to have implemented a graph database, but possibly without the graph walking capabilities of graph databases.

EdgeQL lets you do arbitrary joins as well. Here's how you could compute salaries of employees by department even if you for some reason don't have a link between Employee and Department:

    SELECT Department {
        name,
        employees := (
            SELECT Employee { name, salary } 
            FILTER Employee.department_name = Department.name
        )
    }

Links in EdgeQL are merely an abstraction over the fact that most joins in a well-normalized schema are done over primary keys.

How do I join tables 1:1 without creating a subfield, i.e.:

table1: id, name

table2: table1_id, level

Desired result:

{id, name, level}

The most direct implementation of what you want would be:

  SELECT (Table1.id, Table1.name, Table2.level, Table2.table1_id)
  FILTER Table1.id = Table2.table1_id;

(This has a somewhat annoying extra output of table1_id, which could be projected away if necessary.)

If you want to use edgeql's shapes but still keep the essential join flavor, then something like:

  SELECT (Table1 { id, name }, Table2 { level })
  FILTER Table1.id = Table2.table1_id;

To make it a little more concrete, you can try this on the tutorial database (https://www.edgedb.com/tutorial)

  SELECT (Photo {uri}, User {name, email})
  FILTER Photo.author.id = User.id;

The tutorial database uses links, and so instead of having an author_id we have author.id, but having an author_id property would work just fine---except that then you'd have to do all the joins manually.

You do joins when you traverse schema type paths in EdgeQL, it's just that the actual joins are implicit (well, compared to SQL):

  SELECT User {
    email,
    preferences: {
      name,
      value
    }
  } FILTER .id = '...'

E.g. in the above example we'd join the underlying User and Preferences tables for you.

You can also do cross joins and all other funky stuff.

How would you represent a self join?

If given a recursive schema like this:

    type Tree {
        property value -> str
        link parent -> Tree
    }

you'd traverse the link as usual:

    SELECT Tree {
        value,
        parent: {
            value
        }
    }
    FILTER .parent.parent.parent.value = 'foo'

If there's a need to self-join on an arbitrary property, then you could use a `WITH` clause to explicitly bind the two sets:

    WITH
        T1 := Tree,
        T2 := Tree
    SELECT
        T1 {
            similarly_valued := (SELECT T2 FILTER T1.value = T2.value)
        }

Thanks!

Example from the cheatsheet:

``` WITH P := Person SELECT Person { id, full_name, same_last_name := ( SELECT P { id, full_name, } FILTER # same last name P.last_name = Person.last_name AND # not the same person P != Person ), } FILTER EXISTS .same_last_name ```

Is supporting MSSQL completely out of scope in not too distant future time frame?

Currently out of scope, sadly.

The actual website: https://www.edgedb.com/

Apache 2 license: https://github.com/edgedb/edgedb

Hey, after 1 minute of looking at it, it looks very interesting (composable etc), especially after reading recent sql rants like [0]

[0] https://news.ycombinator.com/item?id=27791539

Thanks! You can also read our own rant here: https://www.edgedb.com/blog/we-can-do-better-than-sql (We can do better than SQL, it was on the front page of HN a couple of times).

Hmm 'select' before 'filter'. This is one of the things that's clunky in SQL; you have to select what you want before you know what you have.

You select the collection here, then specify the fields. The opposite of SQL.

It's interesting that you know SELECT being first is clunky, without understanding why.

SELECT in SQL is clunky because it comes before FROM and JOIN. Do you see FROM or JOIN anywhere here? No.

The collections you're selecting from are literally written before the fields you select.

The real WTF for me is this claims to be a relational database and you can't join seemingly.

I replied to another similar comment here: https://news.ycombinator.com/item?id=27795685

That seems a bit snarky to me?

In any case, the collection you're selecting from is presumably the filtered one, not the original one. Or am I misunderstanding the semantics?

The tech looks impressive and interesting.

How will you guys make money? As a prospective user, what are the chances of being rethinkdb'd?

Cloud & enterprise solutions.

There are a few reasons why RethinkDB failed; there's an excellent blog post written by the founder. I think there are many areas where we are quite different:

- We are building on top of Postgres, so we are not investing too much time into the hardest technical problems like creating robust DB storage and query planning/execution engines.

- Instead we focus our time on designing proper high level APIs, the EdgeQL language, migrations, and the DX.

- Yet still, EdgeDB is a relational DB; it's strictly typed and fast. Which is still what the majority of companies want.

When will EdgeDB be production ready? Also will it handle large scale deployment with Fail-Over and Replication?

> Also will it handle large scale deployment with Fail-Over and Replication?

Built-in support for that is coming, but it's already possible to run EdgeDB on top of Amazon RDS, for example. Ultimately EdgeDB is built on top of Postgres, so it will support replication and fail-over and many of the existing deployment strategies.

For me to use EdgeDB with RDS I would need to compile it [1]. But I don't think this is recommended usage of EdgeDB and there isn't enough documentation on using EdgeDB with an external postgres instance.

I see that I could use:

edb server --postgres-dsn

But I guess you'll release more docs on using EdgeDB with external instances soon, maybe before 1.0 release.

[1] https://www.edgedb.com/docs/internals/dev

edb server is mostly a development-mode proxy to edgedb-server, which is the actual server binary and supports the same options. The Docker image README [1] has some documentation on the supported options.

[1] https://github.com/edgedb/edgedb-docker#official-dockerfile-...

Do you ever intend to replace it with a custom backend? Just asking due to the article yesterday that said the only way to start is with an abstraction over an existing RDBMS, but could potentially evolve later.

Unlikely

We plan to release 1.0 in a month or two. The upcoming beta 3 (in 1-2 weeks) will be the last beta before the RC.

That said, EdgeDB is already stable and solid thanks to the amazing Postgres that we're building it on top of. It's definitely ready now to play with and learn.

What I really like about Prisma et al is that they provide me with a native client. I don’t have to think about my queries as strings, I have autocomplete for everything that can possibly be part of the select/delete/update (in typescript at least).

Now that I’ve gone there I don’t think I can go back, even if this looks otherwise amazing.

Agreed, Prisma with TS is really nice. I’ve recently been using Blitz, which integrates Prisma and Zod (amongst other things) and it feels like so much of the old webapp plumbing is either now gone or is practically writing itself through autocomplete.

Blitz feels like beta software, but I agree it makes the whole React single page development experience really nice.

Yeah, Blitz does feel beta-y, in a true sense: a few things are still being polished, but it’s pretty stable and v1 seems to be close to done.

I’ve only been using it for a personal project, but since the fix for the Material UI recipe (which I think was actually related to an npm behavior) I haven’t run into a single bug.

I suspect that’s because it seems like a lot of Blitz is unifying some really good established libraries into a framework. It’s not written from the ground up.

Yeah, we'll have it all for EdgeDB too.

Ok, this comes at a right time.. I was just starting a new project and considering Neo4j. I think it is unfair to compare it to Postgres but the EdgeQL make it look like the nested queries are easy to execute and comes with graphql support too. SO are there any pointers to how much production ready it is? How does it compare to raw SQL queries and nested queries? thanks. Also could not find much on authentication and authorizations. are there any pointers/

EdgeDB is very close to be production ready. We will release 1.0 in a month or two and we're based on Postgres, so the hardest part of storing the data reliably is there already.

EdgeQL queries are typically way shorter than corresponding SQL equivalents. You can see comparisons and benchmarks here in this blog post: https://www.edgedb.com/blog/edgedb-1-0-alpha-1

Authentication currently is very basic. We plan to have a relatively short release cycle, so in 3-4 months after 1.0 we'll have more robust access control, including read/write policies of individual types and their properties/links.

> The easiest and preferred way to install and set up the EdgeDB server is using our command-line tool.

On the main page. I wish devs stopped giving command line instructions that execute remote bash scripts without any way to look at what's happening under the hood. It's a recipe for disaster, and really, really bad practice.

Everything is open source and we explain what the command will do. And on the Install page there are manual installation instructions + we have a docker image. How else can we improve things?

Edgedb seems very nice.

Is there any plan currently to give back to postgress? If this is already the case, was there anything going upstream from your team?

Yes, we contributed a few patches and bug reports over the years. We are using vanilla Postgres since it hit 12 and now contains our fixes (or fixes proposed by us).

Great! Thanks for the response

I'm thinking about learning SQL. Would it be a sane decision for me to learn EdgeQL instead, as my first introduction to databases?

If I were you, I would rather start with SQL. It is so pervasive that it would never be a skill you regret having. I don’t believe you would regret learning EdgeQL either, or any skill for that matter, but in terms of priority I would suggest SQL would be a safer bet and help you contextualise other querying approaches down the way.

If you decide to go the EdgeQL route first we have some great learning materials, e.g. here's a book about EdgeQL: https://www.edgedb.com/easy-edgedb

What makes this product an “edge” database?

Presumably it refers to edges in an object graph [0]. EdgeDB seems to be currently deployed as a management, schema, and graph-to-sql translation layer for PostgreSQL.

[0] https://www.edgedb.com/blog/edgedb-a-new-beginning#edgedb.

> Note that this SQL query is not very efficient. An experienced developer would rewrite it to use subqueries

Anyone care to explain why is that? I thought the query plan should be the same for JOINs and subqueries.

There is a difference in how large a set `array_agg` would need to process, since `LEFT JOIN` may produce a very sparse relation and NULLs still have to be scanned over.

Correct!

It's not a database meant to run on the 'edge'. I'm guessing it's a graph database built on the PostgreSQL codebase.

> EdgeDB is a relational database that stores and describes the data as strongly typed objects and relationships between them.

> EdgeDB is built on top of PostgreSQL, inheriting all its core strengths: ACID compliance, performance, and reliability.

Does it have something like SQL's

  select *

or do you always have to specify which keys you want?

We plan to implement splats soon, e.g.:

  SELECT User {
    ...User
  }

would you be able to use EdgeQL to introspect an existing postgresql database and run queries on it?

Asking as I have a project ongoing where I don't really have the time to recreate everything in EdgeDB, but wouldn't mind spending a few hours to check how the query language works compared to SQL.

No, this isn't supported, sadly. We might support this mode eventually, but the ETA is unknown.

Would you have any documentation on how to install EdgeDB and connect to an existing postgresql database then?

From browsing the current documentation, I can't seem to find anything similar to sqlalchemy's:

  meta = MetaData()
  meta.reflect(bind=someengine)
  users_table = meta.tables['users']

Looks interesting. How does it compare to Dgraph? Both in features and in performance?

Its not very clear if rust is supported. Please please support rust bindings :)

Not yet. Our CLI is written in Rust and we have crates for the protocol and other pieces, but their APIs are not stable yet. It is on our roadmap though.

Why did you choose Python for your servers implementation language?

Would be cool if you could also run normal postgres alongside

You can. It is possible to point EdgeDB at an arbitrary Postgres cluster and have it coexist with other databases.

Do you have more info about how to do this?

You can run a normal postgres DB alongside EdgeDB. You can use a regular Postgres cluster and create your pg databases in it and at the same time point EdgeDB to the cluster

    docker run --name edgedb -e EDGEDB_PASSWORD=secret \
    -e EDGEDB_POSTGRES_DSN='...' \ 
    -d edgedb/edgedb

RedCrowbar answered this question here [1]

[1] https://news.ycombinator.com/item?id=27815177

> EdgeDB, a relational database with strongly typed schema

Relational database without relational algebra wouldn't be useful, would it? So... uhmm. How do we join? There's no JOIN statement.

I replied in another comment: https://news.ycombinator.com/item?id=27795685

Will EdgeDB have a http api? I want to use postgres with a serverless app but there are practically no options

It already has it. In addition to the binary protocol you can shoot GraphQL or EdgeQL queries at it via HTTP.

Thanks. I am very interested now.

Reply on news.ycombinator.com