But it will take a long time to mature. SQLite the project cannot switch languages, sadly. The only way to migrate is that another project would grow beside in the meantime, mature, and become a viable and compatible alternative.
But, tangentially, if not SQLite, what would you expose as a DB interface in a browser? SQL, while large and hairy, is both powerful and logical (in the relational part, not syntax). A plain KV store like dbm is no match to it. A Redis-like store is better but still more limited. Kdb's approach is sort of wonderful but geared towards time series and not much general-purpose. Is there an existing interface / language you would reuse to give the browser a rich database interface?
Only if either:
- there are anyway some major changes comming
- or major problems/improvements in maintainability and better external dev commitment/support
could rewriting it make sense IMHO.
Lets be honest many (most?) of the "recent" (~3years) security bugs sqlite had would most likely not have been prevented by using rust, go or similar.
EDIT:
I guess the biggest drawback of C is that it keeps contributors away, but I also might keep some of the contributors projects like sqlite might not want to have away. So depending on the maintainer it might be seen as a benefit not a drawback.
Using SQLite's CVE list from 2020[1], we see 12 vulnerabilities:
* Two NULL pointer dereferences; it's impossible to produce a NULL reference in safe Rust.
* Two integer overflows; Rust makes these harder, but not impossible.
* Three UAFs; these are impossible in safe Rust.
* One uncategorized segfault; there are impossible in safe Rust barring environmental constraints (like a stack overflow from unchecked recursion).
* One segmentation fault from incorrect object initialization; this is impossible in safe Rust.
* 3 SQL and table-level bugs; Rust is unlikely to have helped with these.
By my count, that's 6/12 bugs are would be impossible in plain old safe Rust, and another 3/12 that would likely be prevented by normal best practices in Rust. I still don't think this necessarily means that we should drop everything and rewrite SQLite in Rust, but the raw numbers don't back up the claim that doing so wouldn't eliminate the actual security bugs that SQLite is seeing.
[1]: https://www.cvedetails.com/vulnerability-list/vendor_id-9237...
But would a sqlite port limit itself to safe rust?
(As a side note wrt integer overflows, rust only makes it easier to detect them during tests and provides neat methods for integer overflow aware code, but just that.)
Anyway I'm surprised that there where more "preventable" bugs then I expected.
I suppose they wouldn't have to, but why would they bother with unsafe? They proudly announce how few system APIs and syscalls they depend on, so they have no need for that (assuming they chose to not use any number of safe wrappers). Complicated self-referential data structures, perhaps, but that again is the kind of thing that could be exhaustively tested and tucked into a safe interface.
And yes, you're absolutely right about integers: Rust itself is not going to save you in release builds. But it does avoid a major source of overflows in C (implicit conversions and promotions), and has explicit, fallible APIs that are easy to enforce as a lint.
Wouldn't they need unsafe in order to interface with the outside world via their libsqlite persona? Which, I think, is how most of the world consumes their sqlite - embedded in some other language as a library?
You can for example have a C API accepting a Option<&mut A> if A is repr(c) and sized. This is possible due to the guarantees &mut A has about it's representation (if A is..) and the guarantees `Option` has about it's representation wrt. non-null values.
While such pattern exposes a often fully safe rust C-ABI API it has some drawbacks mainly that C is required to make sure the pointer they pass in is valid (null or aligned, allocated and initialized).
You can take this a step further an only accept a `&mut A` BUT that is a bit dangerous as now C calling rust with a null-pointer leads to unsound rust, as rust assumes &mut A is never null.
Another gotcha is wrt. owned heap allocated data, as you need to make sure the same alocator is used for allocating and de-allocating it. Again nothing new, but in C you often doge that issue. (E.g. if you allocate, and initialize a *A in C and then pass it to rust as a Box<A> you need to be very careful that you never drop that box in rust). Through Rust is currently getting support for keeping track of non-default/non-global allocators in the standard library, and you could always setup a global/default allocator which defers to libcs malloc/free.
But more seriously: Rust’s toolchain comes with linting built in, and the community as a whole is much better about applying and responding to static analysis results than the C ecosystem is. And that’s even before we get to false positives, which (subjectively) C static analysis tools seem to spit out a great deal more often.
And I say all of that as someone who’s currently doing whole-program static analysis of C and C++! It’s not that you can’t do it, it’s degrees of ease and a culture of stringency that’s lacking.
BUT, keeps improving == keep changing.
For a library like sqlite, it need to maintain compiler compatibility. It is difficult to track a "live" language.
I have used rust since the 1.0 release and while there had been some churn in the earlier releases even that was fast to fix.
By now it's nearly always new improved clippy analytics/warnings, but clippy is not rustc but a (the) external linker, so this isn't really rust compiler churn IMHO.
Even when there was a bit of churn in the early stable releases it was often remarkable easy/fast to fix.
If there where not trivial to fix things it was nearly always dependencies, not because they are bad but because if you have enough dependencies sooner or later one of them is no longer maintained or deprecates some functionality you need or do a new major release you should upgrade to. But even there it's much less painful then the experiences I had with e.g. npm and the JS ecosystem.
The only really painful thing I did run into was ring/rustls during the time ring yanked all previous dependencies no mater if they where broken or worked fine and still frequently updated some of their core parts. That was no fun, but fully the fault of ring, and it's no longer doing this by now.
There's this idea floating around in some circles that if you could just adopt technology X, more people would contribute to a project.
I myself was guilty of this ... believing that if we added a web frontend to the DAW I've worked on for 21 years, all those JS/webtech developers would show up.
There's no platform or language you could choose for SQLite that would increase the number of contributors. C is already a wildly popular programming language, with far, far more practicing users than Rust. Rust may be The Cool New Thing at present, and in time it may possibly grow to something much bigger, bigger even than the C/C++ universe. But that's not true right now, and it also wouldn't be true even if SQLite was implemented using some impossibly performant JS or its cousin.
I'm sensible to the issues of tradeoffs (e.g. platform support which is a completely fair issue) and manpower requirements and whatnot, but I don't see why it can not switch language. Converting libraries "inside out" has been done. Adding Rust support inside sqlite's build system and migrating modules is technically feasible (again, not opining on whether it would be worth it).
I mentioned that but i don't think that's the subject as it does not warrant such a drastic assertion that sqlite can not switch language.
There's no particular reason to think that this will ever happen. SQLite runs on a lot of crazy stuff that Rust won't want to bother with.
If can switch the officially endorsed "primary" implementation, but only after an alternative implementation has been around for a long time and was battle-tested, all the while the original C implementation continued to exist and develop.
At a purely technical level (ignoring issues of platform support and all) it pretty much can do that, actually: integrate XYZ into the build system, build new features in XYZ, start converting old features to XYZ, end up with an sqlite in XYZ.
That is useless to sqlite itself, this exists only because of go's issues and is essentially unusable outside of the go ecosystem.
The only issue I see with cgo is that they decided to follow such path instead of a proper FFI declaration support like most languages.
Even Java is now having such capability thanks Project Panama.
Issue’s cgo doesn’t just make the build more complex and more expensive (and yeets cross-compilation), it also makes programming way more difficult as go tooling doesn’t understand C and C tooling has no clue about Go (because non-standard calling conventions and insular tooling community and everything), and for all that cost you get little because the go/c call overhead is stupendous (still, though smaller than it used to be), and of course it locks up the scheduler (after a slice of the goroutine being blocked the go runtime will create a new one, but still…).
E.g. the hugo project built with and without extended features. Build without and it’s all go, it compiles in the blink of an eye. The tooling in go, from a devops point of view, is surprisingly good. It’s not just compile speed.
Build with all the c deps and well you’re compiling more code so of course it’s going to be slower but it’s disproportionately so.