SQLite: Vulnerabilities | Hacker News Reader

SQLite: Vulnerabilities

sqlite.org

108 pointstosh34 comments

It is a bit strange that the SQLite developers recommend SQLite as an application file format [0], but at the same time warn against opening database files from unknown sources [1]. In fact, their advice on using the `integrity_check` and `quick_check` pragma work against the performance advantage of using SQLite, as the entire database file now has to be read and parsed.

SQLite definitely still has advantages as a file format, but it would nice if there was bit more consistency in their messaging.

[0]: https://www.sqlite.org/appfileformat.html [1]: https://www.sqlite.org/security.html#untrusted_sqlite_databa...

I find the point sensible. SQLite is a fine internal storage format for an application.

Of course if that storage is swapped among untrusted users, additional scrutiny is needful.

It totally is a sensible point. In fact, _all_ file formats, SQLite or not, has to deal with untrusted data, which will involve some kind of integrity check.

I had some rather large SQLite databases so I just wanted to get a rough idea of the impact:

    Size (MB)   Quick (s)  Integrity (s) 
       8         0.061       0.47
      76         0.39        1.8
     582         2.8        13.
    2926        14.         79.

I tried my best to do the measurements with a cold filesystem cache, but this is by no means rigorous. For example, the databases for this test are roughly similar in schema and they all consist of many small rows. Few big rows might perform very differently.

My first impression was that this page was oddly defensive but, on the other hand, their criticisms of the CVE system made a fair amount of sense. Do others tend to agree with those critiques?

Yes, CVEs are often exaggerated and sometimes just plain wrong. The bulk of them tend to be created by people that gain somehow, either financially or reputationally by creating them, and have no affiliation with either the person who originally discovered the bug or the upstream project that fixed the bug. Such people have every incentive to maximize impact and no incentive to do due diligence. And the review process for CVEs is entirely inadequate to the volume of the task.

Yep - CVE's are sometimes overblown - particularly if they have a codename attached to them which became the trend once it entered the conscious.

Yes, a lot of CVEs follow the form explained in the SQLite article, where you have to already have access to "gain access" in an unnecessarily complicated way. For example, `rm` has a vulnerability where if an attacker runs `rm -rf ~` on your machine, they will delete your home directory.

I think the key problem is that people misunderstand what CVEs are.

Ultimately CVEs are numbers given to a certain vulnerability. It helps so two people know they're talking about the same thing. A typical usecase would be that you have a security scanner that tells you your server is vulnerable to CVE-xxxx-xxxx and you can check the changelog of your server software which version fixes that.

A CVE does not mean that a vulnerability is particularly important. It doesn't mean it's relevant for your use case. And a lot of CVEs does not mean the person who found them achieved something of significance.

From a security perspective it makes sense to treat vulnerabilities that are minor and only relevant for rare use cases (or in combionation with other bugs) as vulnerabilities. But when you start counting vulnerabilities as achievements this distorts things. Plenty of shady security companies and research institutions making "statistics" about CVEs don't help either.

I think the tone of this page is frustration that this page needs to exist. I think if I had to constantly deal with stuff like this I'd be pretty frustrated too:

> CVS-2019-19317 is an excellent example of a bogus CVE. This CVE describes a bug in an unreleased development version of SQLite. We were working on the new generated columns feature of SQLite, and a third-party hacker found an error in the code under active development, and then wrote a CVE against it. The error was fixed before the problem was ever released, and yet still there is this CVE sitting out there, unresolved, and as far as I can tell unresolveable.

Lot's of great related discussion from drh in this[0] sqlite forum thread.

[0]: https://sqlite.org/forum/forumpost/0e8b92001245dec1

It applies to a lot of software actually, I've found that one of the implications of developing in the open is that people feel free to take whatever version they want and treat it the same as a release. The DVCS appears to have exacerbated this because people so often just attach their scripts to pull from "master" instead of respecting tagged releases.

I've more than once discovered an in-development version of a library I'm working on appear as a package in Debian, for example. Suddenly I'm feeling responsible to maintain backward compatibility with an unreleased version because it's now "released" as far as users go. Of course what happened is that I asked the maintainers to remove it and then they never packaged any new versions of my software because of the interpersonal friction it caused.

> Grey-hat hackers are rewarded based on the number and severity of CVEs that they write. This results in a proliferation of CVEs that have minor impact, or no impact at all, but which make exaggerated impact claims.

Makes sense.

Reminds me of a story about people growing snakes so they can then kill them and report the killing and get a reward for it.

When a measure becomes a target, it ceases to be a good measure (common paraphrase of Goodhart's law)

It's the cobra effect: https://en.wikipedia.org/wiki/Perverse_incentive#The_origina...

You probably mean the "Cobra Effect", a form of perverse incentive[0]:

The term "cobra effect" was coined by economist Horst Siebert based on an anecdote of an occurrence in India during British rule. The British government, concerned about the number of venomous cobras in Delhi, offered a bounty for every dead cobra. Initially, this was a successful strategy; large numbers of snakes were killed for the reward. Eventually, however, enterprising people began to breed cobras for the income. When the government became aware of this, the reward program was scrapped. When cobra breeders set their now-worthless snakes free, the wild cobra population further increased.

[0]: https://en.wikipedia.org/wiki/Perverse_incentive

There is the exact same story about rats in Indochina during the French occupation. In fact the stories are so similar that one wonders if they were not specially crafted to demonstrate the incompetence of the colonial rulers.

They are so similar because they show a basic principle behavioral economics.

Yes, maybe. Or, they are parables.

I read somewhere that when scholars offered per-piece bounties on ancient manuscripts found in caves, the herdsmen scouring the caves would drive up their profit by cutting up the finds.

Incentives work.

I found it funny their security page is titled "Defense against the Dark Arts"

https://sqlite.org/security.html

It's actually "Defense Against Dark Arts" (no "the" and slightly different capitalisation), which makes it slightly less of a match for the Harry Potter reference that I assume is the source of the humor.

It seems to have been called both. Formerly, as you say without the "The", but currently the title agrees with GP. Without a last updated date it's hard to know when this changed, though.

Complete history: https://www.sqlite.org/docsrc/finfo/pages/security.in

The title was updated today in response to one of the posts above.

To me this means two things: SQLite's testing has been so effective that most remaining bugs are probably consolidated to a specific category, and yet SQLite is not free from such bugs and C is to blame.

That seems accurate.

A fair critique of C in sqlite would also question whether C is a key factor in sqlite's success.

I don't know the answer to that, only that "X hinders" has to be weighed against "X helps".

Yes, I would say it was still an excellent choice for SQLite to use C. Still we are increasingly questioning the tradeoff offered by C and the case of SQLite gives one (strong) data point.

> Still we are increasingly questioning the tradeoff offered by C and the case of SQLite gives one (strong) data point.

If only we knew whether it is a strong data point pro or contra C.

Note also that the SQLite devs, chasing 1% perf improvements every year, continually rewrite huge chunks of the entire library, sometimes replacing entire components. Their comprehensive testing ensures that there's no API or ABI breakage, and of course the fuzz testing does catch many bugs, it also ensures that SQLite always has a constant flow of new bugs.

A huge class of vulnerabilities are not applicable since SQLite states it’s not to be used for client/server database use cases.

Essentially, if SQLite is not recommended for use which allows remote user use cases - there’s a lot less to worry about.

https://sqlite.org/whentouse.html

We fix bugs and some weeks later there is a CVE for it with only inaccurate information. Hmm...maybee it would be a good idea to fix a bug and help the rest of the world to keep track of it by submitting a CVE as developer himself. Then this can't happen. But no, we think it's a better idea to critize all existing CVE's on our website as bogus.

Relevant discussion about "CVE Stuffing": https://news.ycombinator.com/item?id=25612429

Apparently it is possible to dispute a CVE. It might make sense to dispute all the bogus ones in the hope that this would reduce the incentive to generate them in the first place.

It is difficult and often not successful. You might get a note added saying “disputed”.

It says why it was disputed right in the description. Example:

* https://nvd.nist.gov/vuln/detail/CVE-2017-17688

Reply on news.ycombinator.com