Show HN: Pg-extend-rs – Easy Postgres extensions in Rust

As warning, there’s still a bunch of work to translate various types from Rust to Postgres Datums.

I’m interested in feedback on the library, so if you use it and experience any issues, please let me know.

TiKV (https://github.com/tikv/tikv) is a database (KV-store to be precise) written in Rust and speaks MySQL protocol.

Toshi (https://github.com/toshi-search/Toshi) is Elasticsearch-like engine that uses tantivy (Lucene-like), both also written in Rust.

Noria is an interesting one: https://github.com/mit-pdos/noria

It's compatible with the MySQL wire protocol and offers extremely efficient, incrementally updating materialized views.

I expect extensions in Rust will be much faster than those written in pgSQL, which would need to be interpreted, of course this should be validated. In addition to that, this is stdlib code in Rust, and stable, so you should be able utilize anything from the Rust ecosystem in an extension built in Rust.

There's always a trade-off of adding new languages to a project, and for simple things, Rust might not be the best choice.

PostgreSQL has several bindings, including Python and JavaScript (V8).

You might want to do complex value manipulation at the DB-level, like a regex-like operation, or processing XML.

A lot of Postgres extensions are designed to directly integrate external information systems that PL/pgSQL would never have access too such as announcing certain types of changes to RabbitMQ, or ensuring an ElasticSearch index is fresh with a subset of the data.

Additionally extensions can be used to extend PostgreSQLs native functionality (like PostGIS does).

That's a funny comparison. The much more obvious contrast would be to writing a Postgres extension in C.

Or are you asking, "why have native DBMS extensions at all?" Looking at the set of native Postgres extensions that currently exist would probably be enlightening:

• PostGIS, for one. You can't really define those types—and especially, the algorithms that operate on them—efficiently in PL/pgSQL. In this case, the extension is native for low overhead operation.

• uuid-ossp, for another. To generate the data it does, this extension needs access to several pieces of data from the OS (e.g. the MAC address of an Ethernet interface) that you can't access from PL/pgSQL. In this case, the extension is native to query native OS data APIs.

• Citus, for a third. Citus basically changes Postgres's runtime "operational architecture" in several fundamental ways. It augments the Postgres Postmaster, changes how replication works, etc. None of this is possible from PL/pgSQL. In this case, the extension is native to use native OS threading and IPC APIs to override how Postgres works.

(And that's ignoring the fact that having native extension support usually gets you ABI compatibility with the embeddable runtimes of most other languages, allowing you to then support things like Python or Java—and thus get access to low-overhead shared-memory+FFI-based "IPC" with existing libraries in those languages, rather than needing to have them run as co-servers and communicate over sockets. Which, in turn, gets you the ability to cheaply and easily use client libraries written in those languages to create Postgres Foreign Data Wrapper libraries for Datomic, or Elastic, or the AWS Public Dataset API, or whatever other weird-API-with-relatively-few-library-impls you like.)

Right now, the only additional overhead I can think of would be on transforms from Rust style Strings and &str's to C style CString and CStr's.

Beyond that, I think it should be nearly identical with C/C++.