Hacker News Reader

Comment by mentalgear | Hacker News Reader

PostgreSQL, one of the most popular databases for reliable data management, also does not enable this feature by default.

"By default, data pages are not protected by checksums, but this can optionally be enabled for a cluster. " https://www.postgresql.org/docs/current/checksums.html

I feel it's quite unfair for the title to call out sqlite on checksums, when in reality, as the very closing line of the article states most databases don't do checksum verifications.

View on HN

The submission doesn't seem unfair. It's simply pointing out something for users of a specific database to be aware of, and I don't really think SQLite is maligned or impugned in any way.

And FWIW, Oracle, SQL Server and DB2 enable page checksum storing/verification by default.

A long time ago there was a famous / infamous ad campaign for some food product like bread or milk, and the ad merely stated the true fact that their milk didn't contain any bleach. Of course no one's milk (or whatever it was) had any bleach (or whatever it was).

Specially highlighting something true, but out of context and with no equally special justification, is not an innocent act and is misleading. And yes, absolutely, very clearly, causes harm, and does so unfairly.

A not-unfair version of this same article would just talk about databases, and include sqlite with others, and not only sqlite and not be titled sqlite.

And then there is this: "Hey there I am v. I work at Turso Database."

A frozen banana that WON'T make you sick and kill you.

https://x.com/bluthquotes/status/732749820348096512

Kind of like fake praise / avoiding constructive criticism in the interest of harmony and feelings.

Your milk/bread comparison is specious and invalid. I might say "As an apple enthusiast -- the fruit kind -- I want you to know that apple seeds have arsenic so don't eat lots of them" without having to disclaim every other fruit existing and possibly toxins found in parts of them. If some true-believer apple fan felt victimized, well that would be bizarre, right?

This is a SQLite guy talking about SQLite to SQLite users. They're describing a feature/possible downside of SQLite that users might want to be aware of. They don't need "balanced" coverage of every other DB because it hurts someone's bizarrely fragile feelings. And as I mentioned elsewhere, almost all "enterprise" databases do do checksums by default, if people really want to lean on this "no one does! Leave SQLite alone" argument.

And Turso is literally a SQLite-based firm. This isn't the aha you think it is.

People are just bored of security sensationalism I guess. Too many people want to gain visibility just by reporting either something little-known (but still known) or that needs stars to align in a proper way to be exploitable at all.

Checksums aren't used to protect integrity for security, they're about data corruption.

And that, kids, is why companies pay money to Oracle, Microsoft, and IBM.

Major corporations are not paying exorbitant licensing fees to have checksumming enabled by default. In fact, for enterprises running things like vSAN, ECC DRAM, etc, database checksumming is probably nothing more than additional overhead.

Database defaults in general are a touchy topic. Whatever set of defaults are chosen will be suboptimal for almost any serious user. A far more serious issue is figuring out the actual behavior of a database in different configurations. For instance, Oracle's SERIALIZABLE transaction isolation level only offers snapshoot isolation.

You don’t understand the critical problem that checksums solve at the I/O boundary. PCIe has weak error detection and correction. To transfer your data from ECC memory to your favorite super-robust storage technology requires transiting the PCIe bus, where for a brief moment it becomes relatively easy to corrupt data without anyone noticing. This is the problem that can’t be solved any other way and why checksums are primarily done at the I/O boundary in databases. It is an issue seen in real systems.

PCIe v6 is intended to materially improve the integrity of data transfers but what we are using today is much worse.

> I feel it's quite unfair for the title to call out sqlite on checksums, when in reality, as the very closing line of the article states most databases don't do checksum verifications.

Hi, author of the post here. I work mostly with SQLite and that's the database I'm most familiar with, hence mentioning it in the title. I have also noted at the bottom:

> Again, this is not a bug. Most databases (except a few) assume that the OS, filesystem, and disk are sound. Whether this matters depends on your application and the guarantees you need.

now I wrote a follow up post - https://avi.im/blag/2024/databases-checksum

FWIW, PG's default has been flipped to checksums being enabled by default in the in-development version.

Personally I'm not sure it's the right call, as it comes with some increased write overhead.

Yes, it's a balancing act. I think it's better to have safer defaults here, not prioritizing extra performance. People who are concerned with performance to the degree where checksums would matter need to consult configuration anyway (in many aspects, not only this one) and can disable this specific thing easily.

For the reference, change is in https://github.com/postgres/postgres/commit/04bec894a04cb0d3... (so should land in PostgreSQL 18 in a year, unless it will get reverted for some reason).

Every crumb of information you encounter doesn't have to be subjected to a lens of "but what if the entities involved were sports teams? Would this be "fair" to "them"?"

Reply on news.ycombinator.com