SurrealDB – Document-graph database, for the realtime web | Hacker News Reader

SurrealDB – Document-graph database, for the realtime web

surrealdb.com

Thanks for posting this nerdyadventurer! I'm Tobie founder of SurrealDB. We've got some really big things planned for the database. Any feedback is really welcome!

Hey, random tangential question: Why do you have `user-select: none` on the page? is there a reason that you don't want people to be able to copy/paste any of the paragraphs? Or is there some other reason

(I noticed as I found it jarring since I'll often select text while I'm reading it.)

EDIT/NOTE: Also, I imagine it might somewhat hurt things getting spread by word of mouth. If I wanted to share this with someone I'd naturally copy some sections, paste them into a message to someone that might be interested along with a link to the site. This isn't possibly with your landing page.

Hi mijoharas, no reason to be honest! It's just what we've done for other sites we've built in the past - a design reason more than anything else!

You're not the first one to make this point, so we'll get this removed from our site!

That's a really good note/point/addition mijoharas. We'll get it changed ASAP!

No worries, and no rush or anything. Was really just curious more than anything!

What's the underlying storage engine? I've been doing some design work on a new hybrid LSM/Extendible Hashing system in Rust and am curious what this is using underneath.

Hi packetlost. Currently the underlying storage engines are memory, and distributed TiKV. For the next beta release (coming this week) we also have a local RocksDB implementation and a distributed FoundationDB storage implementation.

In the long run, we want to implement our own key-value store so that we can offer some of the temporal/versioning aspects (which would allow you to view your data in the graph as it appeared at a particular point in time).

For the browser, SurrealDB sits on top of a key-value store which is powered by IndexedDB, so that SurrealDB can run in the browser, using WebAssembly.

My hybrid LSM/Ext. Hash system may be of interest actually. It's purpose built for tracking historical data similar to Datomic's data model, but emphasizing constant-time read operations for current data while keeping the read speeds for historical data reasonable. In short: 2 level LSM tree (log file, immutable page files) with a extendible hashing to index into the page files. The identity of each record is the key + tx instant (time of COMMIT, or some other monotonically increasing number that is unique to a transaction set). Optionally you can index over historical data by having a record per-key that just appends a new tx-instant + offset for better longitudinal query performance.

Hi packetlost this is a really interesting topic to me. I wrote my MSc thesis on this exact topic - https://surrealdb.com/static/whitepaper.pdf ! I've learnt a lot since writing this thesis, and would have many different opinions if I wrote it today, but very interested in this area of research!

Same! I'll read through that paper when I have time. What are your thoughts on how you would implement something like that today?

At the time I implemented the versioning using separate trees on top of the Radix tree. Today I would augment the tree with t he versioned nodes, and would use a Concurrent Adaptive Radix Tree for the base tree. This would allow high concurrency. Then at the base of the tree I would store a temporal link to each tree modification, so that you could access the changes in time order!

Very, very interested in chatting further about this!

Same here! I'll reach out on other channels when I've compiled my notes and have a fully functioning prototype :)

Is your work public? I'm also working on a (loosely) datomic inspired database with pluggable backends, so this might be very interesting for me.

Hi the_duke. The base implementation of that key-value store (written in Golang) can be found here: https://github.com/surrealdb/rixxdb

I don't think the implementation of the ideas in the thesis were added to the public project in the end.

We will be working on this in due course in Rust, however, and the implementation will be completely open source!

I would be really interested to see / find out more about your new hybrid LSM/Extendible Hashing system!

Very cool, love the idea of JS/WASM extensions. In the past couple of years I've seen other new entries in this space - CockroachDB and Planetscale come to mind. What would you say makes SurrealDB unique, if you don't mind me asking?

Hi danielvaughn thanks so much for the comment! We love CockroachDB and Planetscale! With SurrealDB we intended to build a database where you can use the best of the relational database world, the document database world, and the graph database world. You don't have to choose up front how you store your data, but you can use many different concepts to model and analyse your data once stored.

Then with the ability to connect directly form the client application or from web browsers, you don't have to build any API layer or logic layer in front of the database. Your data access permissions and authentication sit right inside the database (only if you want of course), enabling you to build applications quickly, directly against the database.

The multi-paradigm approach sounds fantastic, I've wanted something like that for so long.

As for direct client connections - Firestore uses that approach as well. One issue with them is that your only means of security is the Firestore Rules document, which is a single script that doesn't offer much flexibility. What I've learned from using them is that if you're going to offer a direct client-to-db interface, the rules expression needs to be pretty sophisticated. Not giving advice, that's just my perspective as a potential customer. I'll take a look through your docs and will probably try it out.

What kind of interface and format is used to work with graphs? Many graph queries (your query language looks inspired by Cypher to me) require returning subgraphs, which need to be represented as a graph and not a tree (JSON, XML, etc.). Do you support TinkerPop-like APIs or RDF formats? Or do you use any alternatives (JSON is not such an alternative, of course)?

It would be helpful if people would include [WAITLIST] in the title.

Too many waitlist things going on - I can understand the need for it, but I'm more inclined to think "Scarcity marketing" in most cases.

Of course if some new products couldn't deal with an influx of new users in a timely fashion, better to have a waitlist.

But for the first N people it should be auto-accept.

Hi carvking! The database can be used completely without our own cloud hosted product (for which there is a waitlist on the site). We're still a little bit away from offering our cloud hosted product, but the product itself can be downloaded, tried, hosted, and hacked on without waiting for our hosted cloud version.

We're definitely not into the 'signup to our waitlist' product, just for marketing purposes. It's only there as some users expressed an interest in registering for it, as they didn't want to run SurrealDB themselves. Just to reiterate, all of our hosted cloud functionality is and will be available in the product which anyone can download and use for free :) !

Ah - fair enough, my built in autocomplete needs some fine tuning.

I immediately saw "join waitlist" and came to complain.

My bad - be well :)

Haha no worries carvking! Thank you!

The features are very impressive, but why should I trust SurrealDB with my data over say, PostgreSQL?

Some things I'd like to know more about:

- Testing strategy - Point-in-time restore? - Consistency/isolation model - Performance data - Migrations

For a start, it's blazingly fast!

So is dumping to /dev/null. A trustworthy database needs a lot more than being fast, unfortunately.

Hi phailhaus, good point. We don't want to go for speed (for instance by writing to /dev/null) at any cost, but want SurrealDB to be a reliable and performant backend for any application in the long run. Obviously in our opinion, writing to /dev/null wouldn't make SurrealDB a proper database :) . We have a way to go to catch up with databases like PostgreSQL and other databases in terms of stability and performance, but we will strive to get there!

But is it web scale?

Lol. Unfortunately only /dev/null is truly web scale :) !

Don't make me start using sarcasm tags. Sigh.

Yep especially considering the motto is always databases are slow. Of course they shouldn’t be unusable but the database isn’t meant to be your clients data model

Hi Diggsey, great question. We are currently focussed on functionality and stability, and then will draw our attention to performance. Coming this week we have a RocksDB storage implementation. We've only just launched our initial beta version, and we know there is a lot of improvement and work to be done (some of these performance issues we know about already and are on our Github issues list).

With regards to the consistency/isolation model, SurrealDB sits on top of a number of key-value stores. By using the distributed highly-available TiKV storage backend, https://tikv.org, (and we have a FoundationDB integration in the works), the database is designed to be highly-scalable and highly-available. The same guarantees (albeit just single-node, so no high-availability or scalability) will be available with the RocksDB implementation coming this week. By sitting on top of these key-value stores, SurrealDB ensures that all transactions are ACID compliant. We don't want to go for speed (for instance by writing to /dev/null) over anything, but want SurrealDB to be a reliable and performant backend for any application. Obviously we have a way to go to catch up with PostgreSQL (launched in 1996), but we will strive to get there!

Source : https://github.com/surrealdb/surrealdb

Features : https://surrealdb.com/features

I am developing a web/JavaScript database for several years now [1]. What I do not understand is, how graph based databases, (like surrealDB or gundb) plan to have great performance when they run in the browser. The thing is that you are always limited to store your data inside of the browsers IndexedDB. So as soon as you do not work with the same document-based model similar to the IndexedDB API, you will have big troubles when it comes to performance. For example how can you store and query graph based data in IndexedDB without heavily overfetching either on read or on writes? IndexedDB is already way too slow.

[1] https://github.com/pubkey/rxdb

Forgive my ignorance but What about the file system store? Is that the same thing?

There are 2 things called "file system api" in the browser. One is the "File System Access API" to store files on a users OS. This cannot be used to store data of a database.[1]

The other one is the "real" filesytem api, which is not supported by most browsers. [2]

[1] https://developer.mozilla.org/en-US/docs/Web/API/File_System...

[2] https://caniuse.com/filesystem

Hi typingmonkey. So SurrealDB abstracts away the document aspect of IndexedDB so it can be used as a key-value store. We then use that key-value store (in a similar way to RocksDB) to store the data in SurrealDB. You can see our abstraction layer here: https://github.com/surrealdb/indxdb . It's actually pretty performant in our tests so far, and we'll be releasing this soon as a WebAssembly library, enabling SurrealDB to run in the browser locally.

I know rxdb (am a stargazer) so I'm really interested to hear suggestions or thoughts around this area!

Thank you for these links, I will inspect that.

As far as I know, accessing IndexedDB from WebAssembly only works by passing the data through the JavaScript layer (correct me if I am wrong). So from how I think WebAssembly works, it is likely not any faster then directly doing that in JavaScript. I mean, the storage layer does not have to do any heavy computations, so tunneling the data would be more expensive then what you get from plain js.

When you store the data into an IndexedDB based key-value store, how does the querying work? Do you still use IndexedDBs secondary indexes when a query only matches a range of documents, or does the K-V store has any method to support querying a subset of the data without fetching the whole database state?

Yes you're right. Accessing IndexedDB from WebAssembly needs to go through JavaScript. The only benefit from our perspective is that building it in Rust enables us to use the same logic and code that powers our entire database platform. From a performance aspect, you're right, it still needs to go through JavaScript.

So we don't use any features from IndexedDB at all really. We literally treat IndexedDB as a key-value store interface like RocksDB or something like that. So all of the indexing is done within SurrealDB, and is stored in IndexedDB as a set of binary keys->values (Uint8Array if I remember correctly).

When we want to query a whole table, or a subset of a table, SurrealDB determines what individual keys to load or what key ranges need to be loaded to perform the query.

Would love to chat further about this - it's a really interesting area of discussion for me!

Ok now I understand.

You can come to the RxDB discord for a chat if you want. I am really interessted in using WebAssembly for browser side storage.

I look forward to the day where someone reimplements IndexedDB via Webassembly, with the exact same API, so that we have a faster IndexedDB which uses the old one as storage layer :)

Awesome I'll see you on the RxDB Discord!

Somehow I don't see people getting as excited with new programming language releases compared to database releases. Maybe that is because databases require less effort to integrate in your stack?

It's because there are a lot of good languages but no one single great "surreal" database. (I'm not saying the SurrralDB is "the one").

As you said, there are a lot of good languages, but not a single great one. Similarly there are multiple good databases, but not a single great one.

How does it compare to ArangoDB which is also a popular open source document-graph database? What features does it add/remove?

Hi satvikchoudhary! It's a good question. Obviously I know of ArangoDB, but I don't know it well enough to comment on the main differences, so I'll focus on what SurrealDB is aiming to achieve instead.

SurrealDB is aiming to be at the intersection of relational, document, and graph databases, whilst still remaining simple to use with an SQL-like query language, for developers coming from the relational database side. We are only at the beginning of the journey, but SurrealDB is designed to be run embedded, or in the cloud, with the ability to query it directly from a client application or from a web browser (and only access the data that you're allowed to see).

With our native client libraries (coming soon), SurrealDB will be able to be embedded within Node.js, WebAssembly, Python, C, and PHP applications, in addition to running as a server.

We wanted to create a database that people didn't have to manage, so they could focus on building applications, not the infrastructure. We wanted users to be able to use schema-less and schema-full data patterns effortlessly, a database to operate like a relational database (without the JOINs), but with the same functionality as the best document and graph databases. And with security and access permissions to be handled right within the database itself. We wanted users to be able to build modern real-time applications effortlessly - right from Chrome, Edge, or Safari. No more complicated backends.

I'm not sure how all of this compares to ArangoDB, but happy to learn!

Product looks great. I totally feel the need for new open-source-but-not-for-AWS-to-steal-my-product licenses; however I'm not sure about that vague license for database would work for you.

                      A “Database Service” is a commercial offering that
                      allows third parties (other than your employees and
                      contractors) to access the functionality of the
                      Licensed Work by creating tables whose schemas are
                      controlled by such third parties.

So, if as a service provider, I control the schemas can the customers query the data using your SQL? Can I just collect data, put them to database and proxy their SQL to DB? i.e I'm the co-founder of Resmo (https://www.resmo.com) where we let customers run SQL on their cloud/saas assets configuration, can I use yours? The current wording seems to let me because we don't let any customer to define their schema, resources schemas are defined by us, for now.

It's not obvious to me that they're trying to stop you. This reads as "AWS/GCP/Azure don't steal our work" while otherwise trying to be as wide a license grant as possible.

I'm sure they'd love to sell you support services, but many of these database companies do that by convincing you there is additional value, instead of trying to use licenses. I think because licenses create a scary risk that the db company might increase prices or even stop offering them altogether and you have no real recourse.

Thank you CSDude. We wanted SurrealDB to basically be open source, but with the only limitation of not being able to provide a Database as a Service platform. So in a business or enterprise use, there is no limit at all. You can run SurrealDB with as many nodes as you want, and as many users as you want; you can provide a hosted database internally, or to employees, contractors, or subsidiary companies. The only limitation is providing a paid-for, hosted, SurrealDB database platform.

After 4 years, all of our code becomes licensed with Apache 2.0 license.

In addition, all of our libraries, client SDKs, and many of our core components are completely Apache 2.0 or MIT licensed (https://surrealdb.com/opensource).

We have a whole page describing the license here: https://surrealdb.com/license!

We would definitely like to chat if you are interested in using SurrealDB!

Benchmarks and comparisons to other popular databases? Landing says no complicated new languages but introduces SurrealQL?

Would be nice to see a Jepsen test

Hi endisneigh. We are currently focussed on functionality and stability. There are a number of changes to our code that we know about (https://github.com/surrealdb/surrealdb/labels/performance), that we want to make really soon, and then as soon as those are complete, we'll run some benchmarks against other databases, and detail some comparisons.

It is however hard to perform these direct comparisons, as the functionality is pretty different to a lot of the database out there, but nevertheless it definitely is something we want to get to in due course!

The database's feature set is very interesting !

One thing to note is the license, which is incompatible with OSI open-source licenses : https://github.com/surrealdb/surrealdb/blob/main/LICENSE

Hi lovasoa thanks for your comment! We did try to stay as open as possible, but unfortunately the license (as you noted) is not an actual OSI approved license. We've got a whole page about why we chose this license, what the limitations are, and how we are contributing to open source (https://surrealdb.com/license).

Effectively though we didn't want to limit the database usage in anyway except for preventing people from offering a paid-for hosted SurrealDB-as-a-Service. Some database companies offer a 'core' open source database, and then put their enterprise features behind an enterprise license (or close source them completely). We wanted to build in the open, but at the same time it is our intention to offer a cloud database version so that developers can get going really easily in due course. This doesn't mean that developers have to use it though :) !

Let me know if you have any other questions or points!

I appreciate that and fully understand, but, in many places, having a non standard licence pretty much guaranties it can't be used. I can't imagine volunteering myself to engage with legal to get this approved.. I don't need this is my life..

Hi jpambrun we're definitely open to comments and suggestions around this!

I imagine it was rhetorical, but as you probably imagined I have no suggestions. I understand your needs and imperatives, but they are unfortunately incompatible with mine. Getting a green light on using custom licence is near impossible. Procurement and contracting for database as service is also a nightmare.

I think mongo and elastic could get away with it because they already had captive users. For illustration, I didn't really look at your offering, I went to the licencing section and just summarily disqualified. I am also unlikely to use it personal pet projects, because the learnings cannot be leverage in my paying job.

No not rhetorical - was genuinely interested in any suggestions. Out of interest, if the product was split (like other database products) into a 'core' open-source version, and an 'enterprise' limited license version, would this make any difference?

Depends what is behind the custom license and how well it's communicated. If it's only the db as service management magic I wouldn't mind. Anything more and I would worry I would miss out on upcoming features few years down the line.

Why wouldn't putting only some enterprise features under your "business" license work for you ?

I don't think anyone will want to make a business out of the database you wrote until it gets massively popular. And it won't get massively popular if it's under a proprietary license.

Hi lovasoa, this is definitely an option, and we're open to suggestions in this area!

Great ! I would love to be able to use a version of SurrealDB, even with a more limited feature set, in my open-source projects.

Ok lovasoa, we will definitely give this a thought!!! Just out of interest (and so I can understand your position / usecase better), how would the current license prevent you using it in any open-source projects at the moment?

Well, I could, but a free software with non-free dependencies is not of much use...

Would you use another software with the same licensing restrictions as SurrealDB in SurrealDB itself?

I've been seeing a lot of projects lately that use Rust for database-related work. How can someone get started writing a database these days?

Hi techn00! It's definitely taken a while for us to get to this point. We initially starting planning to build SurrealDB back in 2016, and started building on top of our initial database prototype (built in Golang) back in 2018. Over that time we've made many changes, both to the system design, and to the code, and architecture. it's just the beginning of our journey, and we've got some really stuff planned.

> How can someone get started writing a database these days?

Take my class. It starts next week: https://15445.courses.cs.cmu.edu/fall2022/

15-445/15-721 got me a job. Thank you so much for making them publicly available.

I wonder how this would pair with Elixir & LiveView. My initial thought is that it would work together quite nicely.

Hi mackeyja92 great suggestion! We're not big Erlang/Elixir users, but we definitely want to integrate with Elixir and LiveView in due course! We actually received a suggestion to do this just yesterday!

We would love to hear any suggestions or ideas in this area. Let us know in whatever way you want (https://surrealdb.com/community)!

Looks very interesting. Kind of reminds me of RethinkDB with its real-time features. Was any inspiration from them?

Absolutely mackeyja92. We were users of RethinkDB back in the day. We wanted to bring the best features/functionality of NoSQL databases (MongoDB/RethinkDB/CouchDB/...), graph databases (Neo4j/OrientDB/...), and relational databases (MySQL/PostgreSQL/...) together into one platform, and then add some awesome functionality on top of it!

In addition we really wanted the realtime functionality to be available when connecting directly from the browser (like Firebase), and that's something that RethinkDB didn't offer.

Realtime is undoubtedly the dream... What restrictions or limitations are present for live queries?

How do you plan to be sustainable? RethinkDB was promising but it didn't last.

Hi dodyg, it is our intention to provide a fully-managed cloud hosted version of SurrealDB. We'll also offer support to those companies who want/need it. But our primary business model will be the DBaaS platform. We definitely don't plan on going down the same route as RethinkDB did!

I'm glad to see a team with a new-ish approach in the DB world... I like the object-like querying! and the JS embedding.

It really looks to me like SurrealDB could become big in the future, ambitious feature set.

DBs take a lot of time to mature, so I want to congratulate you and wish you the best

Thank you very much weddpros! My brother and I have been building it and building apps on top of it for 7 years now. We launched SurrealDB in public open beta 4 weeks ago. Just the two of us at the moment! We have some really big things planned for SurrealDB.

Thanks so much weddpros! We definitely have some ambitious plans for SurrealDB, and we have lots of exciting stuff to come!

Very interesting license structure: https://github.com/surrealdb/surrealdb/blob/main/LICENSE

Hi gardenfelder, we've got a whole page on our site explaining the thought process and our intentions around the license - https://surrealdb.com/license !

This is really cool. So do you have your own storage engine for the single node case, and you're using TiKV for the distributed one? Or is the local storage case also built on an existing engine like RocksDB?

Hi hiyer! We've got a single-node storage implementation using RocksDB coming this week. With regards to running in distributed mode, TiKV (and soon FoundationDB) will be able to be used for storage.

In the long run we would like to build our own single-node key-value store (based on a Concurrent Adaptive Radix tree), but this is a little way off still!

> soon FoundationDB

This is super. Finally there's a reason to use FoundationDB on ordinary systems.

Thanks dodyg. The implementation is already in the codebase. We're just looking to get it built into the Linux version in the next release!

Many warning signs with this. From ridiculous claims to license. I'm out.

Hi AtNightWeCode, we're just a team of 2 developers at the moment, and completely bootstrapped. This is only our initial release, and we have a long journey ahead of us. We realise there is lots of work to do. It would be great to get an understanding of what you mean in more depth :) ! With regards to scalability and high-availability, in distributed mode you run SurrealDB on top of TiKV (and soon FoundationDB), so we don't claim to provide this functionality ourselves (yet).

With regards to the license we wanted SurrealDB to basically be open source, but with the only limitation of not being able to provide a Database as a Service platform. So in a business or enterprise use, there is no limit at all. You can run SurrealDB with as many nodes as you want, and as many users as you want; you can provide a hosted database internally, or to employees, contractors, or subsidiary companies. The only limitation is providing a paid-for, hosted, SurrealDB database platform.

After 4 years, all of our code becomes licensed with Apache 2.0 license.

In addition, all of our libraries, client SDKs, and many of the core components used in the database are completely Apache 2.0 or MIT licensed (https://surrealdb.com/opensource).

The license is not really the big issue.

Very cool… live queries are a game changer!

What’s the testing story? Do I need to connect to your servers in order to do e2e testing? Any plans to offer a mock server or anything like that?

hi erikpukinskis, obviously SurrealDB can be run on your local development machine, or in your own cloud, so you would never have to connect to our servers for testing. In addition, we are releasing a number of client libraries which (in addition to connecting to a local or remote database server), will also allow you to run SurrealDB embedded i the language of your choice.

Coming soon are Nodejs native module (with embedded database functionality), Python (with embedded database functionality), WebAssembly (so you can run SurrealDB completely in your web browser), and C (with embedded database functionality).

In due course, we want to have native integrations for PHP, Java, and Swift too (and then more to come down the line).

This will mean that you could run tests without having to connect to any external service/platform/server!

Let me know if you have any other questions!

Cool. Any insight to how Surreal DB compares to Cockroach DB?

Hi geenat, CockroachDB is an awesome database! SurrealDB and CockroachDB are quite different, in that CockroachDB is effectively a highly-available distributed PostgreSQL database (aiming at that similar functionality), while SurrealDB is aiming to merge the relational / nosql / document / and graph databases together. Hence the functionality is quite different. To make SurrealDB distributed we have built it on top of TiKV (on a side note TiDB which is built by the creators of TiKV is more of a competitor to CockroachDB than SurrealDB is).

We will definitely in due course provide some documentation and comparisons with many of the other databases out there, and also we intend to do benchmarking with the other databases as soon as we have implemented a few of the performance issues that we know about (https://github.com/surrealdb/surrealdb/labels/performance)!

This looks pretty awesome! My biggest question is support for mobile. I wonder if there’s any planned integration with React Native?

Hi catchmeifyoucan. We definitely plan to support React Native in due course. We are also happy to take any suggestions/contributions/ideas in this area to make it a really great integration.

I really like that the features page is so transparent about what's completed/in progress/planned

Thank you very much BankHottas! My brother and I are building it in the open and want to be as transparent as possible. We have some really big things planned for SurrealDB that aren't on our features page, but which we will be adding very soon!

Publish Code on GitHub proclaim your love to Open Source... but actually use Proprietary License...

Hi PeterZaitsev, many of our core components that are used within the database are open source under the MIT / Apache 2.0 license. With regards to the core database code, it is free to use with no limits on number of nodes, data size, number of connections, number of connected users, features, or functionality. The only limit is for providing a paid-for hosted database-as-a-service. In addition, after 4 years, our complete source code for that version release becomes Apache 2.0 licensed. We've tried to keep this as open as possible for developers / businesses / enterprises to use, whilst at the same time protecting ourselves from some of the events that have happened in the database world over the last few years. It's definitely not our intention to just publish our code on Github just to pretend to be open source!

On the other hand, you are able to use all of the libraries that you use only because they didn't make the same choice as you, and chose truly open-source licenses...

In particular, the scaling abilities of your product seem to be based on the capabilities of TiKV, which itself is based on RocksDB. These are both databases in and of themselves, and if any of these two had adopted the same license as you, then you wouldn't have been able to release the product you are releasing...

Now this looks great, except for the lack of time travel and branching that keeps me to TerminusDb.

This might be trivial to some, but I kinda hate it that I can't select text on their homepage.

Hi tobiemh, will there be any plans to add native support for Svelte / Sveltekit?

Hi yawnxyz - absolutely we intend to have support for Svelte in due course! We aren't Svelte users ourselves, so any suggestions or ideas are very welcome! Do let us know in the community - https://surrealdb.com/community

Will you add support for connecting from IntelliJ/DataGrip?

Hi jxi thanks for the question! Our plan is to make SurreealDB accessible and usable from as many codebases/platforms/architectures as possible.

We've got plans for integration into all of the big IDEs down the line, so we absolutely will add IntelliJ and DataGrip to the list.

We'll definitely want to get as much feedback in this area when we get to it, so do join the community (https://surrealdb.com/community) as we love all ideas and suggestions!

Does it support http webhook on table events ?

Hi revskill, absolutely you could do something like this:

DEFINE EVENT changed ON person WHEN $before.email != $after.email THEN http::post('https://my-webhook.com', $after);

It's very flexible! We're also releasing support for the fetch() function within the JavaScript functions soon, which will enable you to do the same thing from within the embedded JavaScript functions!

How does this compare with Supabase?

perhaps I'm daft, but why should I care that you've written it in Rust? The number of times Rust is mentioned on the landing page feels...odd.

I love shiny new things. However, without some sense of how this compares to alternatives, it's difficult to see why I should dive down the surrealdb rabbit hole. Feature lists are ok, but what I really want is context. Why should I care about what surrealdb can do?

Hi RayVR, being built in Rust definitely isn't a main selling point, but it has made building SurrealDB easier. We initially built our prototype in Golang, but decided to re-write it in Rust so that we could improve on certain aspects that were hard to do with Golang.

That aside, we wanted to create a database that people didn't have to manage, so they could focus on building applications, not the infrastructure. We wanted users to be able to use schema-less and schema-full data patterns effortlessly, a database to operate like a relational database (without the JOINs), but with the same functionality as the best document and graph databases. And with security and access permissions to be handled right within the database itself. We wanted users to be able to build modern real-time applications effortlessly - right from Chrome, Edge, or Safari. No more complicated backends.

It's only the beginning of our journey (we're just a bootstrapped team of 2 at the moment), and we welcome all feedback!

> certain aspects that were hard to do with Golang.

Do you mind sharing those aspects?

Absolutely arunc!

You can actually see the Golang version - it's on a separate branch in the main repository.

We had written our query parser from scratch (which would read character by character). This worked well, up to a point, but was hard to extend. Building in Rust we've used the nom crate for parsing, and it has enabled us to do some pretty clever and advanced stuff with the SurrealQL query language (futures, embedded JavaScript functions, while at the same time remaining performant.

Other things we wrote from scratch (at the time), were our own binary serialisation framework (similar to MsgPack, but with slight differences and performance improvements). We then had to write our own serialisation logic on top of this, so that each data type would serialise and deserialise efficiently. With Rust, using the Serde crate, this is done completely differently, and allows us to have the same performance (or better), but with no custom writing of data tags!

In addition, many other improvements, especially around generics mainly. It made building SurrealDB much quicker, but it also means that all data passed around throughout the system is able to be reasoned about easier. Instead of brute-forcing with the race checker, you really have to have a concrete idea of how and where your data will be used. This starts out harder (fighting against the borrow checker), but in the long run it makes writing safe and performant software so much more pleasant.

I'll try and go into this in detail in one of our future blog posts!

Fantastic, thank you!!

Would love to hear about the pain points you had with go.

Absolutely debuggerpk. My answer to arunc above goes into it in slight detail! We will definitely plan to do an in-depth look at the decision to transfer from Golang to Rust in a future blog post!

In the context of "History doesn't repeat, but it rhymes", this is just what happened when Java appeared. Everyone noting it was written in Java (as, believe it or not, it was the cool new language).

It might have actually been worse, because there was a strong penchant for naming products starting with a "J" to indicate the fact (i.e. JNotepad, JDatabase, etc).

It might actually be a good marketing technique to get other Rust aficionados to try your product. But otherwise, there isn't any real value now that "write once, run anywhere" doesn't just belong to Java anymore.

Even Javascript rode on the Java hype/marketing albeit having no correlation with it.

No real connection, but amusingly I think javascript ended up delivering on java's "run anywhere" pitch at least as well as java did in the end.

I can run javascript in a query to this DB apparently for instance, I can't do that with java.

From the operations point of view: Rust code is usually fast, safe, free from data races, easy on RAM, possible to optimize very well, compiles to native binaries (easy to deploy). Unlike C, it's easy to test and provides good abstractions, so the code generally will be less buggy.

From the point of view of a developer: when it comes to extending the database, writing an extension in Rust might be a pleasant and welcome experience. Rust interfaces with C/Python/Javascript easily and compiles to WASM, so the code written in it can be easily reused on the front end.

What's not to like?

Hi dmytrish that's exactly it! We are really close to releasing our WebAssembly driver (built in Rust), a native Python driver which can connect to and embed SurrealDB (built in Rust), and a native Node.js driver which has the same functionality as the Python driver (built in Rust). The plan is to do the same for the C driver in due course too.

The simplicity of what you described means that we can release a common set of libraries with similar functionality, but using a single common library.

It mentions rust 3 times on a very very long landing page, and once is in a list of languages it has bindings for. That's really not "the number of times" your comment implies.

It is worth mentioning in my opinion mostly because that you should care that it isn't written in

- C/C++, because of security concerns

- Java/Scala/<JVM family> because embedding these languages (and this is available as an embedded database) is a pain, also because of common (though not inevitable) memory bloat.

- Python/Ruby/..., because of common performance issues (DBs being performance sensitive software)

Obviously there is perfectly good software written in all those languages, and terrible software written in rust. The the odds of it being good however are substantially increased when you find out it's written in rust because for this domain, out of the common programming languages, rust really does have the best set of tradeoffs. A landing pages goal is to give you as many signals that "this software might be worthwhile" as possible, and being written in Rust (or really not being written in pretty much every other common programming language) signals that right now.

> It mentions rust 3 times on a very very long landing page, and once is in a list of languages it has bindings for. That's really not "the number of times" your comment implies.

I count 7 times. Once with the cargo install, once "built entirely in Rust", once "built with Rust", three times quoted under "Loved by Developers", and once under "Database Libraries and Clients".

Did the link change? I just read through that landing page and didn’t really notice Rust mentioned. It’s mention a bit on the GitHub readme, but that would be expected as getting started with the code?

Also, people should care about the language something is written in, especially in OSS, as that implies a number of things. Specifically, what the ecosystem of that language is like in terms of maturity, how easy it is for you as a developer to get started with the project, what the build tool chain is and will work in your environment.

I know I disqualify projects that I will look at purely based on the language and toolchain it uses since I know it will be harder to integrate into my companies workflows.

Reply on news.ycombinator.com