Hacker News Reader

Comment by ec109685 | Hacker News Reader

While technically true as written, it seems to downplay the significance:

> The bug is a data race with tight timing constraints. It is unlikely to occur in common use.

A large customer did experience this corruption, so it's important for people with tailscale's setup update immediately.

> The developers have never been able to reproduce the bug organically and had to add special testing logic to SQLite that deliberately triggers the circumstances of the the bug in order to verify that the issue has been fixed.

View on HN

Odd not to highlight the sentence where they answer the obvious question "Why Tailscale in particular?":

> They also explained why we were more likely to hit the bug than other SQLite users: we take manual control of the checkpointing process, and we checkpoint very aggressively. Even a bug triggered by a rare condition was bound to hit us eventually.

The bug requires two checkpoints in very quick succession, which presumably isn't something sqlite would do on its own, as it would be a pointless waste of performance.

Tailscale isn't mentioned in the SQLite changelog.

Quick note that data corruption bugs that are impossible to reproduce are not uncommon (perhaps they're the norm). So some amount of head scratching trying to figure out a plausible scenario by which the system could get into the state represented by the smoking remains is often required. Then you attempt to force it into the supposed bad state by modifying code paths accordingly. So the approach used in this case is clever, but it's not particularly unusual in the world of data stores.

I agree it's tricky, but Tailscale had downtime by this bug and the SQLite Changelog downplayed it.

How is "Fix the WAL-reset database corruption bug" downplaying this? https://www.sqlite.org/changes.html

> The bug is a data race with tight timing constraints. It is unlikely to occur in common use. The developers have never been able to reproduce the bug organically and had to add special testing logic to SQLite that deliberately triggers the circumstances of the the bug in order to verify that the issue has been fixed.

I read that and it feels like some theoretical thing versus causing impactful outages for workloads shaped like tailscale’s.

Reply on news.ycombinator.com