LibSQL is an open source, open contribution fork of SQLite | Hacker News Reader

LibSQL is an open source, open contribution fork of SQLite

github.com

38 pointsrmason67 comments

This was posted yesterday, complete with an inflammatory blog post.

This fork has exactly zero new code, but they already managed to change the license and the code of conduct. Their plans include io_uring, Rust and WASM functions - the first two negate the "single-.c-and-.h multi-platform" paradigm that made SQLite what it is, the third thing seems like a feature very few people would care about. They also want to make SQLite distributed for some reason (uh, just use Postgres?)

I think making sqlite distributed-capable is very interesting. Postgres isn't trivially distributed and is a full fledged database server that's not appropriate for scale to zero workloads, though others are working on that as well.

These do all sound like interesting ideas, but in our current Cambrian explosion of new databases (especially "X open-source SQL database you know and love, but distributed" forks) - you can fork SQLite and build that without being needlessly confrontational about it, or including the language about how they'll rejoin SQLite if their terms are met, which makes me feel like this is a fork about leverage as much as realizing a different vision for a database.

I imagine they love the sqlite codebase but are not ecstatic about having to maintain a fork and would like to have sqlite upstream their changes such that they don't have to import patches in perpetuity. This is a valid frustration that most commonly happens with OSS that's built by a corporation and thrown over the wall. I think in this case the dig at the end of the README is a way to be like "look at those religious cult members, closed off from accepting contributions" which is a cheap shot but all's fair in politics. A lot of companies build on top Linux, PG and others and are able to reach consensus with the community.

Postgres is not a distributed database.

Sure, it isn’t. But most things do not really need a distributed database, just something usable from multiple locations, and Postgres solves that problem easily.

Most vehicles don't need rocket propulsion, yet you can't go to the moon in an FSO Polonez ;)

Just because you, personally, don't have a good reason for using a distributed database doesn't mean there aren't folks who do!

We thought that, built an app on Postgres, got huge, now we wish there was a distributed Postgres vendor on AWS. I’m grumpy Azure bought Citus. Maybe Aurora is trustworthy now? Like 3 years ago I’d still hear about it eating peoples data.

City's is fully open source now.

Sure - but it'd be nice to have it managed for us.

Are you talking about this blog post? https://glaubercosta-11125.medium.com/sqlite-qemu-all-over-a...

What part(s) is/are inflammatory? (I'm honestly asking because I don't see anything obvious and I wish to understand your perspective. I'm not trying to imply that there's nothing inflammatory there by using a question)

I am guessing that "SQLite does not accept contributions" and "the industry is going in a new direction" are what most find objectionable.

SQLite cannot accept random contributions. It's on A350 flight systems, where failure is not an option. The maintainers are able to uphold this quality, but random contributors are not.

It appears that the true desire is log shipping (archived logs, transaction logs, etc.) - the ability to move transaction data between databases, perhaps in multimaster.

The only way such functionality is getting into SQLite is if the project agrees to write it, likely for major funding.

A fork implementing these features will be regarded with interest, but will not be accepted without substantial rework.

How is that inflammatory? SQLite not accepting contributions is their own policy. Saying that the industry wants different features... is that inflammatory? This new fork wants contributors, so of course this is going to be a distinction of their project.

Linux is used to run the world and there are failures as well as contributors.

There might be more subtlety than is immediately obvious.

It is implied that SQLite is not run properly because it is not responsive to many of its users. User contributors will be sorely disappointed.

There is an assertion that SQLite is behind the times. It has a laser focus on reliability, which hold back features desired by some.

The article disregards the TH3 testing compliance, and asserts that forks are inevitable, which is not true. TH3 is a moat around SQLite that dissuades such attempts, and ignoring this fact also likely invokes umbrage.

There is also a mention of the code of conduct that SQLite has chosen. It does not inspire universal admiration.

The article wants the following, and some are unlikely to ever appear upstream:

1. We would like to use io_uring and asynchronous interfaces

2. We would like to provide replication hooks for distributed systems

3. We would like to allow for WASM-based user-defined functions

4. We would like to use eBPF

You don't need to fork SQLite to get log-shipping replication:

https://litestream.io

Multimaster?

> SQLite cannot accept random contributions. It's on A350 flight systems, where failure is not an option. The maintainers are able to uphold this quality, but random contributors are not.

NASA uses nodejs and kubernetes. Everyone's safety threshold and threat model is not the same.

https://mobile.twitter.com/gitlab/status/1174427375347666944...

Does the FAA concede the sunset of DO-178B?

> SQLite cannot accept random contributions. It's on A350 flight systems, where failure is not an option. The maintainers are able to uphold this quality, but random contributors are not.

Do you think Airbus updates their copy of SQLite to the newest version constantly? Bugs make it into SQLite every release, and no released version of SQLite is known to be DO-178B compliant. Seems a lot more likely that it is as Dr Hipp said a few years ago [1], that some customers have qualified specific versions of SQLite in specific configurations for use in applications that require DO-178B compliance. He says that he thinks that those companies certified it to D or C level, which is non-critical stuff, like failure leading to passenger discomfort, e.g. they may be using SQLite in the in flight infotainment system.

> Their plans include io_uring, Rust and WASM functions

Awesome! That sounds kick ass! Turn up the perf some orders of magntude, use a safer kanguage with better tools, and make it usable safely from anywhere!? My heavens this db could go to all kinds of new places!

> the first two negate the "single-.c-and-.h multi-platform" paradigm that made SQLite what it is,

Seriously feels like c needs to grow past keeping everyone rubbing two sticks together to eek out cross-compatibility. We've been stuck at such a silly minima in c/c++ for so so long. Why is code not reusable?

Also it helped, yes, but im not sure this simplicity is the defining "lite" thing of sqlite. Yes: that is damned lite to use/consume, abut it's just one characteristic mong other lite-nesses. It doesnt strike me as that essential.

And if it's impedeing exoloration & growth & other uses... then heck yes a breakaway attempt should be formed. There's excellent material here, & limiting it to a tiny quaint shape & form, purely to help the ridiculous mess of build systems/package management (and especially their absences), seems like an extreme tradeoff. For sure there should be ventures to try to explore & go on, see what the other untapped values might be!!

> the third thing seems like a feature very few people would care about.

Well, i'd hope there's a lot of people who wouldnt think to care today that might tune in & turn on, if they had a lightweight db system that was easy to embed. I dont know how many billion devices have access to the web, but giving them the ability to use sqlite seems sharp & obvious!

Yesterdays post on qemu painted a great picture of a similar case, of there being a ton of excellent materials here, but it being locked into a very conservative & narrow perspectuve of use. The breakout of qemu-kvm unlocking trapped potential, rewriting the world, seems apt. Progress ought have representation!

For context, this is a continuation of a story from yesterday:

"SQLite: QEMU All over Again?" https://news.ycombinator.com/item?id=33081159

And previous light discussion for this same submission link: https://news.ycombinator.com/item?id=33082414

It's funny how for years SQLite has been moving along constantly, stable, with new features, but all of a sudden when it's popular, people seem to want to have a piece of the credits (looks good on your resume!)

It was forked by a startup with devs/founders who've worked on QEMU and ScyllaDB, among other things. I don't think they're chasing resume credits. :)

> We take our code of conduct seriously, and unlike SQLite, we do not substitute it with an unclear alternative. We strive to foster a community that values diversity, equity, and inclusion. We encourage others to speak up if they feel uncomfortable.

No, you don't. You strive to foster an ideologically homogeneous "community" where the language police has a virtual "license to kill". I much prefer the SQLite "code of conduct" [1] which at least allows dissent without being shot down due to an accusation of -ism or -phobia.

[1] https://sqlite.org/codeofethics.html

That is a lot of effort to complain about SQlite's Code of Conduct.

I mean. I was definitely unfamiliar with their "Code of Ethics" [0] until I read it and I cannot say that I'm a tremendous fan. But also, it's the absolute last thing mentioned in the readme, so unless you have additional information that I don't, I don't know why you think that's the main reason they forked, especially when they tell you all the reasons they forked and that isn't really one of them.

[0] https://sqlite.org/codeofethics.html

It was described yesterday as content extracted from St. Benedict's Rules for Monasteries (which is a much larger book).

I read it years ago, but cannot confirm.

He goes out of his way to mention something peculiar despite its non-relevance to the technical features of the project, and does so again on a Medium post introducing the fork. In fact the only interesting said at all is about the code.

(As to DRH's Code of Ethics... well, the product speaks for itself.)

The sum of all things speaks for itself. We have an excellent product and a written statement about how all current developers are bound by a religious covenant. And no contributors accepted. If you want to create a fork and welcome lots of contributors, then I don't see how the CoC as written would stand. Of course this has to change.

> That is a lot of effort to complain about SQlite's Code of Conduct.

Anyways, given that the super-majority of the blog post was about features and the different directions that sqlite is being pulled in, and the only singular mention of the CoC was a paragraph at the very end of the blog, I don't really see the summarization of the blog post as just about the CoC as honest.

This is one of those things that's not really a huge in of itself in that it doesn't have much practical implications. But it's such a wildly inappropriate and unprofessional thing to do that it raises questions in my mind about using SqlLite. Which is a pretty impressive thing to do given how good the software is and how widely it's used.

Why is it inappropriate? A lot of OSS projects make political statements all the time, I see no difference at all

I dispute that they do but regardless a political statement is not the same thing as a profession of religious tenets.

It's certainly unusual, but all it's saying is that the founder and current dev team hold those religious tenets. It's (explicitly even) not forcing it on others.

Wearing a polka dot bowtie is unusual. This crosses the line into being inappropriate for a workplace.

It should be obvious why, but if not consider a future where the SQLite team needs to replace a member. What's the plan if the best candidate is an Orthodox Jew? Are they going to update the CoE to read:

>The founder of SQLite and all current developers, except for Ruth Cohen, have pledged...

Even if they're the nicest and most welcoming Christians in the world and will make a better change without hesitation it's still presents a disincentive for non-Christians. There's simply no place for that in any professional setting.

>>it's still presents a disincentive for non-Christian

For some non-Christians.

Some of us non-Christians are open to reasonable people of every persuasion.

> still presents a disincentive for non-Christians

The CCCoC presents a disincentive to non-progressives or progressives who just happen to disdain identity politics and the race to the bottom in search of ever more fine-grained intersectional levels of oppression that involves.

In a truly "diverse" society a political statement would carry the same weight as a religious one.

I consider the 'Contributor Covenant Code of Conduct' to be far more politically charged than the 'rule of Benedict' as used by SQLite. The CCCoC opens the door wide open to identity politics where the 'rule of Benedict' with 'special emphasis to 'rule 71 in particular and also rules 2, 8, 9, 18, 19, 30, 66, and in the spirit of all the rest', i.e.

71: Make peace with your adversary before the sun sets

2: Then, love your neighbor as yourself

8: Honor all people

9: Do not do to another what you would not have done to yourself

18: Be a help in times of trouble

19: Console the sorrowing

30: Do no wrong to anyone, and bear patiently wrongs done to yourself

66: Do not love quarreling

...is clear in its intentions without referring to any identity groups and without proposing a whole list of sanctions and sanctioning procedures. Compare the following preface to the list of 'rules' from the 'code of Ethics' from SQLite:

No one is required to follow The Rule, to know The Rule, or even to think that The Rule is a good idea. The Founder of SQLite believes that anyone who follows The Rule will live a happier and more productive life, but individuals are free to dispute or ignore that advice if they wish.

The founder of SQLite and all current developers have pledged to follow the spirit of The Rule to the best of their ability. They view The Rule as their promise to all SQLite users of how the developers are expected to behave. This is a one-way promise, or covenant. In other words, the developers are saying: "We will treat you this way regardless of how you treat us."

...with the text of the CCCoC:

We pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, and healthy community. ... Community leaders are responsible for clarifying and enforcing our standards of acceptable behavior ... This Code of Conduct applies within all community spaces, and also applies when an individual is officially representing the community in public spaces ... Instances of abusive, harassing, or otherwise unacceptable behavior may be reported to the community leaders responsible for enforcement at community@libsql.org. All complaints will be reviewed and investigated promptly and fairly. All community leaders are obligated to respect the privacy and security of the reporter of any incident. ... Community leaders will follow these Community Impact Guidelines in determining the consequences for any action they deem in violation of this Code of Conduct: ... Correction ... Warning ... Temporary Ban ... Permanent Ban

Take your pick of policies, I've taken mine and will follow the path of Benedict.

Even putting the identity politics bit of it aside, it is just unnecessarily hostile, and comes across as such if you haven't acclimated yourself to things like this already. Plus a lot of it, as near all of it that actually matters, goes without saying.

I would be interested in knowing whether there are real instances that having a code like this one resulted in a better open source product as an innate feature of the code.

>I consider the 'Contributor Covenant Code of Conduct' to be far more politically charged

This?

>We as members, contributors, and leaders pledge to make participation in our community a harassment-free experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, caste, color, religion, or sexual identity and orientation.

>We pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, and healthy community.

Specifically, what part of it do you object to?

>is clear in its intentions without referring to any identity groups

Uh of course if you exclude the parts that do refer to identity groups it doesn't. But when you do include them it's clearly a Christians only club:

1. First of all, love the Lord God with your whole heart, your whole soul, and your whole strength.

10. Deny oneself in order to follow Christ.

21. Prefer nothing more than the love of Christ.

41. Put your hope in God.

42. Attribute to God, and not to self, whatever good you see in yourself.

44. Fear the Day of Judgment.

45. Be in dread of hell.

46. Desire eternal life with all the passion of the spirit.

49. Know for certain that God sees you everywhere.

50. When wrongful thoughts come into your heart, dash them against Christ immediately.

57. Devote yourself frequently to prayer.

58. Daily in your prayers, with tears and sighs, confess your past sins to God, and amend them for the future.

60. Obey in all things the commands of those whom God has placed in authority over you even though they (which God forbid) should act otherwise, mindful of the Lord's precept, "Do what they say, but not what they do."

62. Fulfill God's commandments daily in your deeds.

70. Pray for your enemies in the love of Christ.

72. Never despair of God's mercy.

As stated before and as stated in the 'code of Ethics':

Understand that the developers used an existing document from 1500 years ago, not to transport SQLite back to the year 480 AD but because it encompasses the spirit of community. The original rule of Benedict was written as 'timeless spiritual guidance on how our lives should be an expression of mutual service and love, seeing Christ in all people, particularly the stranger'. Feel free to exchange 'Christ' for the virtues which are normally ascribed to him if the reference feels alien to you, it will not change the spirit of the document which is to strife for 'mutual service and love' instead of 'strife, oppression and conquest'.

Also note that I quoted the sections I did because those are the ones specifically noted in the 'code of Ethics', not to leave out any references to religion. It should not surprise anyone that a document from the founder of a monastic order contains references to religion so I have no problems with those.

As a last point I'd like to add that the rule of Benedict manages to profess true inclusivity - and in my eyes probably too much of it for current society - instead of the Nomenklatura-type of the same ordered by documents like the CCCoC where an ordained group of commissars gets to be judge, jury and executioner over the masses. That is not inclusion but exclusion of all who do not abide by the whims of those commissars. As far as I'm concerned the CCCoC and similarly authoritarian "policies" do not lead to "inclusive communities", they just make contributors walk on their toes when communicating to avoid becoming a target for the language police.

Wow, they took the time to lay out a project roadmap for you and features they're interested in and that's all you took from it? A single, minor mention of it at the end?

The project is a buzzword bingo roadmap (Rust! WASM! io_uring! Distributed databases!), a new CoC and license, and zero new code. No new code means I can’t take it seriously and I doubt they will ever deliver.

> SQlite's Code of Conduct

Well, I had never seen that before. As an atheist that was quite a, well, revelation.

I am not militant in any way and have a generally lax stance about these things. For example, I have had believer friends ask if they could pray over me when I was going through rough times. My response was "I would be honored". No sense being a complete asshole when someone offers something out of love.

The rules page includes this:

> No one is required to follow The Rule, to know The Rule, or even to think that The Rule is a good idea.

> The Founder of SQLite believes that anyone who follows The Rule will live a happier and more productive life, but individuals are free to dispute or ignore that advice if they wish.

In other words, for me, very much my stance with regards to religious belief (or not). It's a "live and let live" position. Something I sometimes feel is sorely needed in today's hatred/outrage-first society.

Yes but the sqlite CoC also says that it's a covenant between all current developers and their users, and for any project seeking contributions that's going to make people consider things. If you were a user, then well, you're a user, so enjoy the one-way covenant. But if you're a developer and an atheist as you say, then I don't see how that covenant as worded would stand.

I see this as similar to a company that says while all its current employees pledge themselves to a religious covenant, it has never discriminated against anyone on the basis of religious faith; and yet all current employees make that pledge. If in fact the wording on the completeness of the covenant is in error, it should be easy enough to point out a sqlite developer who does not make this pledge.

> for any project seeking contributions that's going to make people consider things.

SQLite isn't and doesn't claim to be seeking contributions, so it seems to me that the existing CoC covers 100% of the surface area. If they started seeking contributions, I would agree that they should consider expanding the CoC.

Also, I think it's good to note that the CoC doesn't say that asshole-ish stuff will be tolerated. It's simply quiet on that. I would guess that (much like the vast majority of projects before the Contributor Covenant came about) they would not permit that sort of thing to occur, particularly given the values they've already expressed.

Yes, I'm talking about LibSQL. If LibSQL wanted to make a fork and wants a religious plurality of contributors, then of course they would have to change the CoC. Otherwise people seeking to join would ask, "Wait, when you say all current developers make this pledge..."

Unless I'm misinterpreting what you said, it sounds like you like the SQLite Code of Conduct?

I do as well. I'm an atheist as well and I hate having people's religious opinions forced onto me. I believe firmly in tolerance and live and let live. I really like this Code of Conduct. It doesn't (IMHO) say that you can be a jerk to people and be part of the project. It doesn't say that you have to be a Christian. All (it seems to me) to say is that the people working on SQLite will be good and tolerant, and that the founder is Christian and believes strongly in tolerance. I fully agree, more of this is sorely needed these days.

> Unless I'm misinterpreting what you said, it sounds like you like the SQLite Code of Conduct?

The part I like is where they state, as their official position, that they are OK with people not wanting anything to do with this. As you said, tolerance, live and let live.

I also recognize I am able to exist as an atheist in a Christian nation precisely because of tolerance and mutual respect, with emphasis on "mutual". There are parts of the world where being atheist, LGBT, or <insert group membership here> can easily get you killed.

And then there's the fact that any decent person would agree with quite a few of the items on their list. Here are 46 I think might qualify:

    Then, love your neighbor as yourself.
    Do not murder.
    Do not commit adultery.
    Do not steal.
    Do not covet.
    Do not bear false witness.
    Honor all people.
    Do not do to another what you would not have done to yourself.
    Do not become attached to pleasures.
    Love fasting.
    Relieve the poor.
    Clothe the naked.
    Visit the sick.
    Bury the dead.
    Be a help in times of trouble.
    Console the sorrowing.
    Be a stranger to the world's ways.
    Do not give way to anger.
    Do not nurse a grudge.
    Do not entertain deceit in your heart.
    Do not give a false peace.
    Do not forsake charity.
    Utter only truth from heart and mouth.
    Do not return evil for evil.
    Do no wrong to anyone, and bear patiently wrongs done to yourself.
    Love your enemies.
    Do not curse those who curse you, but rather bless them.
    Bear persecution for justice's sake.
    Be not proud.
    Be not addicted to wine.
    Be not a great eater.
    Be not drowsy.
    Be not lazy.
    Be not a grumbler.
    Be not a detractor.
    Recognize always that evil is your own doing, and to impute it to yourself.
    Keep death daily before your eyes.
    Keep constant guard over the actions of your life.
    Guard your tongue against evil and depraved speech.
    Hate no one.
    Be not jealous, nor harbor envy.
    Do not love quarreling.
    Shun arrogance.
    Respect your seniors.
    Love your juniors.
    Make peace with your adversary before the sun sets.

Nothing wrong with aspiring to be this person, even if people could succeed at a portion of this the world would be a better place.

The set is undefined and contradictory. Even single statements here can be interpreted in a variety of ways.

I suspect this code of conduct would not hold in a court of law, because it has effects not related to this particular piece of software.

Additionally, it is already contradicting SQLite's safety culture. The safety regulation requires being conservative in code acceptance - you do not "let live" here or it can cause deaths.

Activist arseholes forcing their "cancel culture" on others if anything remotely related to pro-western ideas of religion or philosophy doesn't bend to their authoritarian whim.

I for one think the Code of Conduct in SQLite is excellent.

These threads are a great example of the problem with open source. People will complain no matter what. Even if there’s no new code, one could say being on GitHub and being able to track the changes are worth it alone, but alas.

If it works out, great, if not, we’ll SQLite isn’t going anywhere.

> These threads are a great example of the problem with open source. People will complain no matter what.

But this is true of non-open-source projects/products as well, and also everything.

> These threads are a great example of the problem with open source. People will complain no matter what.

Who cares if people complain? If you're writing open source software because you want a hugbox to heap you with unearned adoration, you've made a big mistake. Respect is earned in this field, that's the way it should be. There are no participation trophies; make something good if you want people to thank you for it.

I wish this project good luck but I'm glad how SQLite team kinda does it's own thing like using Fossil for development instead of Github, not accepting outside contributions, very different code of ethics document etc.

The part about the SQLITE code of conduct at the end is unnecessary imo.

Oh, it's very necessary because it's a significant part of their attempt to align SQLite development with modern open source practices. A code of conduct similar to the Contributor Covenant is table stakes for real-world open source development now. It provides guarantees to contributors and users that community participation will be welcoming and inclusive and harassment will not be tolerated.

With so many highly talented developers coming from marginalized backgrounds -- nonwhite, gay, trans, gender-nonconforming, and/or neuroqueer -- not providing these assurances alienates a prime talent pool.

What does that mean, however?

How will SQLite enforce good behavior of a developer, in perpetuity? Will it remove code contributed 10 years ago by now an open bigot? Is it pertinent only to communication platforms related directly to SQLite? Would it void and null a safety certification or audit done by such a person?

These and other questions come to mind when seeing a CoC of such an undefined form. It is important for the document to answer them. This particular one seems to answer that the CoE is irrelevant to anything as nobody can follow these rules, if read strictly. So why have it at all?

The single sentence list can be interpreted in multiple contradictory ways, which is how we got many religions and even monk orders. It is a code of conflict not ethics.

Most importantly, I would not sign this, even if I were not an atheist. It is incompatible with rules of modern Catholic Church, Islam and probably more. It is probably not compatible in a strict form with whichever branch of christianity the founder follows as well.

> Will it remove code contributed 10 years ago by now an open bigot?

Is this any project's policy?

> So why have it at all?

The code of ethics explains why they adopted it. Some customers required suppliers have codes of conduct.

> It provides guarantees to contributors and users that community participation will be welcoming and inclusive and harassment will not be tolerated.

I had no idea harassment and inclusiveness were problems so easy to solve that simply writing a manifesto would suffice. Silly schools for not thinking of it earlier.

Writing a manifesto doesn't do anything. Adopting a Code of Conduct and implementing a compliance board to review violation complaints and take disciplinary action does help prevent harassment.

They want attention but they don't have code and their roadmap of features they aspire to is gagworthy. Complaining about the way SQLite is organized and managed is pretty much all they've got.

It's quite an achievement to create such a toxic fork that it's announcement gets flagged on HN.

I wish this project all the success, and the responsible parties all the notoriety, which it and they deserve.

This project cannot have the particular success of maintaining the high quality of code as the parent without obtaining TH3.

There is an extensive test suite that SQLite uses, known as "test harness 3" (TH3), that allows them to confirm compliance with the DO-178B avionics standard.

Dr. Hipp had previously intended to license TH3. Any startup with goals of maintaining DO-178B quality and compliance should do so.

Otherwise, the fork should be regularly rebased to newer branches to regularly pull new SQLite features.

TH3 is discussed at some length in this interview:

https://corecursive.com/066-sqlite-with-richard-hipp/

Reply on news.ycombinator.com