Comment by nemothekid | Hacker News Reader

Comment by nemothekid | Hacker News Reader

I might be missing something (We use sqlite for our embedded stores) - but I feel like "failing silently" is alarmist here.

1. If the WAL is incomplete, then "failing" silently is the correct thing to do here, and is the natural function of the WAL. The WAL had an incomplete write, nothing should have been communicated back the application and the application should assume the write never completed.

2. If the WAL is corrupt (due to the reasons he mentioned), then sqlite says that is that's your problem, not sqlite's. I think this is the default behavior for other databases as well. If a bit flips on disk, it's not guaranteed the database will catch it.

This article is framed almost like a CVE, but to me this is kind of like saying "PSA: If your hard drive dies you may lose data". If you care about data integrity (because your friend is sending you sqlite files) you should be handling that.

Also, partially applying a WAL has obvious issues even though the author of this post would somehow prefer that. If we update 3 rows in a database and the WAL entry for one of the rows is corrupted, do they expect to ignore the corrupted entry and apply the rest? What happens to data consistency in this particular case?

Even worse: SQLite, by default, does not immediately truncate WAL files, but rather overwrites the existing WAL from the beginning after successfully applying a checksum.

Doing what the author suggests would actually introduce data corruption errors when "restoring a WAL with a broken checksum".

However, SQLite, by default, always truncates the WAL files on last connection close

> When the last connection to a database closes, that connection does one last checkpoint and then deletes the WAL and its associated shared-memory file, to clean up the disk.

https://www.sqlite.org/wal.html

Yes, but you can't rely on an orderly database close when it comes to correctness. Non-clean shutdowns are much more common than actual low-level data corruption in many SQLite applications.

Reply on news.ycombinator.com