Hacker News Reader

Comment by zasdffaa | Hacker News Reader

I did some utterly trivial benchmarks of aggregating over a (large) single table, and found PG and MSSQL to be about the same (both using row storage).

Where does your 100X slowdown figure come from, and what 'architectural limitations' are you referring to? Genuine question as DBs architectures are interesting to me. TIA

View on HN

Postgres and Sql Server are both similar generations of database architecture, so I would expect them to perform similarly in some aggregate sense. For Postgres, I would argue that the major architectural limitations that impact relative performance are in three areas:

- The storage engine architecture is obsolete and highly suboptimal. It has no way to accommodate modern high-density or high-bandwidth storage, both of which are the norm now, nor is it possible to implement any of the (very successful) I/O schedule optimization concepts that have emerged in recent years. The net effect is a gross underutilization of the capabilities of modern storage hardware.

- The internal page layout and table architecture in Postgres is a textbook classic but not appropriate for most workloads on modern hardware. Modern page engines need great performance across diverse categories of workloads and data models -- we use databases for a lot more than accounting systems these days -- while also being amenable to aggressive use of hardware optimization like SIMD. Thinking in terms of "row-oriented" and "column-oriented" is a false dichotomy; optimized page engines commonly have many elements of both but are not identifiable as pure expressions of either nor even a simple hybrid (e.g. PAX layouts). This is one of the easiest changes to backport into old database architectures.

- Indexing in some modern systems have been completely reimagined to great effect. Tables are effectively index-organized with no secondary indexing structures but without loss of high query selectivity across diverse columns; there is a separation of the concerns regarding fast search and key constraint enforcement; major increases in index write throughput concurrent with major decreases in storage and memory footprint (no B-tree bloat). For small tables, this optimization has no effect and may even be a mild pessimization depending on the workload. As tables become larger the performance starts to diverge greatly, becoming multiple orders of magnitude faster. The Postgres architecture can't be modified to support this.

If you built a database engine that was approximately state-of-the-art in these three areas, I would expect a 100x difference in performance relative to Postgres for many large data model workloads using server hardware like an AWS i3en.24xlarge. Obviously building such a database would be a hell of a lot of work, it isn't something you can reasonably do for fun on nights and weekends.

Edit: this comes across rude (I suppose it is), and maybe I'm wrong to doubt you but I'm no DB n00b and pretty well everything about your post just doesn't make sense, given my knowledge.

What? [deleted] A quick browse then...

> The storage engine architecture is obsolete and highly suboptimal

cite?

> It has no way to accommodate modern high-density or high-bandwidth storage

WTF does this even mean? Storage is a hardware issue, nothing to do with the DB engine. And it has mmap and huge pages <https://www.postgresql.org/docs/current/runtime-config-resou...> so what's it missing?

> nor is it possible to implement any of the (very successful) I/O schedule optimization concepts that have emerged in recent years. The net effect is a gross underutilization of the capabilities of modern storage hardware.

No evidence given.

> Modern page engines need great performance across diverse categories of workloads and data models

It's chunks in memory that's all, how is it failing then?

> Indexing in some modern systems have been completely reimagined to great effect. Tables are effectively index-organized with no secondary indexing structures

just what?

> but without loss of high query selectivity across diverse columns

selectivity is a feature of data, not how it is organised, at all

> there is a separation of the concerns regarding fast search and key constraint enforcement

what? what? what? what? what? what? what? what? Search is a read issue, key constraints are a write issue...?

> no B-tree bloat

seriously WTF?

> As tables become larger the performance starts to diverge greatly, becoming multiple orders of magnitude faster

"multiple orders of magnitude" - Wuuuut? cite please

I'm on a long road trip, so can't spend a lot of time responding right now (gotta hit the road). But some quick points:

The page layout is important because many workloads are memory-bandwidth bound. Effective selectivity mechanisms outside the page reduce the need for intra-page selectivity optimization in terms of delivering performance but you still need to preserve memory bandwidth. This biases designs with good page-external selectivity mechanisms to optimize for widening the set of workloads they perform well on instead of squeezing out slightly more selectivity for narrow workloads.

Some of these assertions are self-evident and non-controversial, such as the poor storage performance of Postgres and the issue of B-tree bloat generally. Using mmap() for storage is the low-performance option, articles regarding which are regularly posted on HN, so the fact that it "has mmap" is a good example of why it is expected to perform poorly (though it isn't the only reason in the case of Postgres). I believe there are plans in the works to potentially redesign the Postgres storage layer in future versions, so this may improve at some point.

But more broadly, you seem to be a bit confused about the theory of database kernel design and the tradeoffs that can be made there? You are questioning things about how actual, real, databases, including most open source ones, are designed.

> Effective selectivity mechanisms outside the page reduce the need for intra-page selectivity optimization in terms of delivering performance but you still need to preserve memory bandwidth

"mechanisms/optimization/delivering performance" - these are just aspirational words - exactly what 'selectivity mechanisms'? I can barely make sense of this. In fact, I can't. At all.

> Some of these assertions are self-evident and non-controversial

They aren't self-evident because they aren't evident to me, and as for non-controversial - this is just ducking the question. You're still not justifying anything.

> and the issue of B-tree bloat generally

A btree page will be between 1/2 and completely full. Random insertions should make them ~75% full. In MSSQL I just created a table of ints (clustered PK) and inserted about a million random ints and had a look at fullness (page = 8060 bytes):

    - Avg. Bytes Free per Page.....................: 2046.7
    - Avg. Page Density (full).....................: 74.71%

yep, so an average of 3/4 full. Bit of expected free space. Where's the bloat you talk of, and what's the alternative structure if this is unacceptable.

Edit: this is just leaf nodes. You might claim it's ignoring non-leaf nodes. Check: 4MB data gets you 56K non-leaf. Massive 2,000 factor fanout (8K page / 4 byte ints) is why. So what bloat?

> Using mmap() for storage is the low-performance option, articles regarding which are regularly posted on HN,

for small files, so I understand. For large AFAIK they're good, which is why they're used. So what's the alternative? Point to where HN says mmap is slow for large files.

> that it "has mmap" is a good example of why it is expected to perform poorly

justify this please.

> But more broadly, you seem to be a bit confused about the theory of database kernel design and the tradeoffs that can be made there?

You may be right but you've just handwaved my doubts away without any evidence.

> You are questioning things about how actual, real, databases, including most open source ones, are designed.

I'm questioning your claims in the hope I can understand better. I'm not a total noob. I am wondering why I can't make sense of you posts.

It might be helpful to read something like

DB2 with BLU Acceleration: So Much More than Just a Column Store

https://db.cs.pitt.edu/courses/cs3551/16-1/handouts/db2BLU.p...

Real-Time Analytical Processing with SQL Server

https://www.vldb.org/pvldb/vol8/p1740-Larson.pdf

to get a sense of the gap between postgres/open source databases vs something designed (at least partially) to better support analytical workloads.

Do you have any good sources to seek out for learning about these improvements, such as optimized page engines and better indexing?

> both using row storage > Where does your 100X slowdown figure come from

Given they specify analytical workloads, I'm betting a lot of it comes from column storage.

I need some ELI5 explaining to: colstores allow you to pick out just the cols of data you need rather than get the full row and discard what you don't need.

If that alone explained the 100X speedup that would mean the row is storing 99% of data that's not of interest to a query. That would a) be unlike pretty most table structures + queries I've ever seen, but in those cases cases such as when a fat blob is stored in the row, you hive off blobs to a separate table and only join to that when blob's wanted.

I'm missing something big here.

I doubt it's that alone (and a 10% factor is much more reasonable than a 1% factor). But I think another factor is that column stores usually compress the data in each column. Which can be particularly effective for columns with a lot of repeated values.

Missed compression, thanks!

You might be interested in [0], which is one of the course readings for [1] which has video available ("Storage Models, Data Layout, & System Catalogs"). Specifically, the paper asks if you can turn a row-store into a column-store by just vertically partitioning the schema or building more indexes etc; the answer is no, and they go into various reasons why (late materialization 3x, compression 2x to 10x depending on whether query is accessing sorted data, etc).

[0] https://15721.courses.cs.cmu.edu/spring2020/papers/08-storag...

[1] https://15721.courses.cs.cmu.edu/spring2020/schedule.html

Now that is a bloody great answer! (too little karma to upvote, so my thanks instead).

Try it out yourself experimentally. You'll see the difference quickly.

https://datastation.multiprocess.io/blog/2021-10-18-experime...

Might be missing something, but using an interpreted language that splashes its objects around in memory with pointers everywhere, using json, and gets a 50% speedup, depending, doesn't look like a convincing test of anything.

Do it in whatever language you want to prove the concepts! I'm not trying to convince of the facts. I'm mentioning an approach to experimentally learning the facts yourself. :)

> and what 'architectural limitations' are you referring to?

Row-based vs columnar-based storage. Latter is more suitable (read much faster) for OLAP workloads.

Columnar storage, pre-aggregation, and advanced indexing techniques based on algorithms like bloom indexes, hyperloglog, etc.

hyperloglog is not an indexing method

Reply on news.ycombinator.com