Bridged Indexes in OrioleDB: architecture, internals and everyday use? | Hacker News Reader

Bridged Indexes in OrioleDB: architecture, internals and everyday use?

orioledb.com

76 pointspella15 comments

Cannot wait for OrioldeDB to reach General Availability. Postgres needs options for open-source separation of storage and compute.

That kind of exists thanks to NeonDB? https://neon.com

Althoug with them being recently acquired by Databricks it remains to be seen how the open source version will fare.

> That kind of exists thanks to NeonDB?

This is unrelated to NeonDB. OrioleDB has been acquired by Supabase. https://supabase.com/blog/supabase-acquires-oriole

Neon is indeed unrelated to OreoleDB, but Neon does also provide the separation of storage and compute in Postgres which GP asked about ("Postgres needs options for open-source separation of storage and compute"). A mention of Neon (which is Apache 2 licensed) therefore isn't totally unwarranted.

I understand Neon is open source and I think it’s an awesome product, but apart from the risks associated with longevity of open source once a company gets acquired - although the storage engine is open source, the control plane isn’t and is non-trivial to implement oneself. Orioldedb is positioned as a Postgres extension, which is must easier to setup (even on an existing operating database) than migrating to a completely different architecture that Neon provides.

OrioleDB isn't a separation of storage and compute, its a more efficient storage engine for Postgres to replace the existing HEAP engine. This is like how in MySQL we could swap MyISAM for InnoDB and eventually RocksDB.

I did some benchmarks on it previously to show how much of an improvement it gives over the stock HEAP engine

EDIT: correct link to the public dashboard below, thanks for the heads up @kiwicopple

https://airtable.com/app7jp5t0dEHyDpa8/shr00etqywoDW2N6N

fwiw I couldn't access your airtable link, but I found this one online:

https://airtable.com/app7jp5t0dEHyDpa8/shr00etqywoDW2N6N

thanks for running the benchmarks, it helps to have external parties verifying the progress

There's an experimental feature which separates storage and compute.

https://www.orioledb.com/docs/usage/decoupled-storage

Is the need for Oriole negated by using a system that separates storage from compute like Neon, Xata?

(Neon CEO)

Not really. OrioleDB solve the vacuum problem with the introduction of the undo log. Neon gives you scale out storage which is in a way orthogonal to OrielDB. With some work you can run OrioleDB AND neon storage and get benefits of both.

> OrioleDB solve the vacuum problem with the introduction of the undo log.

Way more than just this!

> With some work you can run OrioleDB AND neon storage and get benefits of both.

This would require significant design work, given that significant OrioleDB benefits are derived from row-level WAL.

Answering on behalf of Xata, it is orthogonal. I'm curious to try out Oriole on our platform when I get some time.

"Bridged Indexes" is a non-standard term. These are just secondary indexes using logical pointers with a mapping index. IIRC, Oracle, Hana, and HyPer do the same thing.

Source: https://db.cs.cmu.edu/papers/2017/p781-wu.pdf (see Table 1 + Section 6.1)

> secondary indexes using logical pointers with a mapping index.

rolls off the tongue.

It's more than just one mapping index, though.

The CMU paper indicates the logical keys are either TupleID or Primary Key, while the bridged index is actually a TupleID that resolves to a Primary Key, which resolves to the actual tuple - one more level than indicated by 6.1's explanation of logical pointers.

Reply on news.ycombinator.com