Hacker News Reader

Comment by adamgordonbell | Hacker News Reader

Host Here. Thanks for sharing.

Richard is a great story teller and adventures that SQLite has taken him on pretty interesting. I think he in on hn as well ( @SQLite).

One interesting thing I found about how SQLite is developed is the amount of testing that is done. Between unit tests and parameterized tests and fuzzer tests they are doing billions of tests per release and spending days running them before each release. I know I don't like to wait for a long CI build before I deploy a service but something like SQLite needs whole different category of resilience.

View on HN

That part really made me laugh. A few months ago, i developed a wrapper in swift for sqlite on a project. A coworker of mine had a bug on what sqlite returned, and i couldn't find anything wrong. He told me "it has to be a bug on sqlite", and i said "no, sqlite has no bugs… but i don't really understand what's going on". And so we posted a bug report on the sqlite forums. I told my friend "if you found a bug on sqlite, on your first try, i'll buy you a bottle of champaign".

Of course, it was a bug with my wrapper (included a \0 char at the end of strings).

I just forwarded him the link to your podcast, saying "we really had no chance"

SQLite is a great way to find bugs in your fuzzer: if your fuzzer finds bug, it’s time to debug your fuzzer!

I enjoyed this episode. I knew about all the tests before, but what came clear in this interview is that Sqlite inc is using their tests as a unique way to keep contracts on Sqlite development flowing. Since they can test it so much better than anyone else, it would be crazy not to hire them, rather than hiring someone else to develop a patch.

While they're keeping those tests proprietary, and relatedly have a closed development model, this strikes me as still better than typical open core strategies for monetizing free software, or at least pointing in better directions.

Agbell, since you're here, could I ask you add a standard link to the RSS feed for your podcast?

> Sqlite inc

Minor nitpick: technically, the primary company that supports SQLite is an incorporated partnership named Hipp [ed note: yes, that's Mr. Richard Hipp], Wyrick [ed note: Hipp's wife] & Company, Inc. dba Hwaci (https://www.hwaci.com/).

Will do!

Done, joeyh.

Thanks, that really is a great conversation.

I'm going to have to dig to see if he describes somewhere why Fossil is such a great fit for SQLite.

Couple things:

* immutable

* built in first-class ticketing system

Happens to use SQLite as its storage engine too (SQLInception?), which brings at least a couple (off the top of my head) features:

* SQL is used to query things, which is very powerful

* the repository is a single SQLite file. Easy to store, copy, and…

* one can checkout multiple working copies into various directories. Or, in other words, I can have three discrete, independent workflows happening simultaneously, but only one copy of the repository backing them all.

It’s also got a built-in server that you can use locally or publish for remote people to access via their web browser that’s essentially GitHub-in-a-box.

Also… (kidding. I’m just going to stop here).

Edit: formatting

Regarding the multiple working copies feature, sounds the same as Git worktrees.

I took a quick scan and: it might be analogous, yes. Not sure if any subtlties apply.

The git workflow in the docs (https://git-scm.com/docs/git-worktree) looks really clunky.

Here’s what I do in fossil:

$ cd ~/work/project_x

$ fossil open ../fossils/projx.fsl (Edit, edit, edit)

$ fossil ci -m ‘descriptive msg’

(Get an idea, but for whatever reason do not want to disturb this directory)

$ mkdir ~/work/radical_idea

$ cd ~/work/radical_idea

$ fossil open ../fossils/projx.fsl <*note here there is no sense of this being an inferior or “sub” checkout*> (Edit, edit, edit)

$fossil ci -m “cool idea” —branch cool_idea # commit work to new branch

(Test..)

$ fossil co trunk #switch branch back to trunk

$ fossil merge cool_idea #merges our work from cool_idea branch into trunk branch

$ fossil ci -m ‘merge [cool_idea] for radical new feature’

$ fossil close # this working dir syncs everything necessary to the repo (../fossils/projx.fsl) and is now essentially “offline”

$ cd ..; rm -fr radical_idea

$ cd project_x (Note: this working directory now not a tip of trunk anymore…)

$ fossil update # now up to speed (Continue hacking…)

Seeing this makes me want to try out fossil now.

It looks to me like the main difference is that with Git you can't have multiple worktrees on the same branch. Apropos of your comment "Note: this working directory now not a tip of trunk anymore…", to Git being on the tip of a branch is a thing of note, so it doesn't ever want to accidentally invalidate another worktree's notes about being on the tip of a branch.

In Git, that'd be:

    $ cd ~/work/fossils/projx.git

    $ git worktree add ~/work/project_x master # add a worktree with the 'master' branch checked out

    $ cd ~/work/project_x

(Edit, edit, edit)

    $ git commit -m 'descriptive msg'

(Get an idea, but for whatever reason do not want to disturb this directory)

    $ git worktree add ~/work/radical_idea -b cool_idea # add a worktree with a new 'cool_idea' branch

    $ cd ~/work/radical_idea

(Edit, edit, edit)

    $ git commit -m "cool idea"

(Test..)

    $ git switch master # whoops, can't do this
    fatal: 'master' is already checked out at '~/work/project_x'

    $ # let's just do the merge in the other worktree, since we're about to switch anyway

    $ cd ~/work/project_x

    $ git worktree remove ~/work/radical_idea # or: rm -rf ~/work/radical_idea && git worktree prune

    $ git merge cool_idea

(Continue hacking…)

> <note here there is no sense of this being an inferior or “sub” checkout>

There is not in worktrees either. You can use worktrees with a bare repository for instance. It's just that for backwards compatibility and the convenience of most users a non-bare clone creates a default worktree. That's exactly what you see if you `git clone` a repository then `git worktree list`, a worktree corresponding to the working copy.

Here is a summary of similarities and differences on Git and Fossil functionalities: https://fossil-scm.org/home/doc/trunk/www/gitusers.md

Also quite interesting the general comparison: https://fossil-scm.org/home/doc/trunk/www/fossil-v-git.wiki

Both sites are written from Fossil guys, so they may not be 100% neutral.

I've migrated to git for my personal projects, mainly because magit (emacs frontend for git) is awesome.

But fossil is a pleasure to work with, especially on Windows, because a) it comes as a single executable that one can just copy to another machine, and b) it does not require ssh or anything else to be installed.

The web interface is amazing, especially considering the size of the executable file.

Fossil have much better command line IMHO. I even started git wrapper to get rid of it's cryptic syntax.

[1]: https://github.com/severak/ginger

Very good podcast, thank you. What I personally missed is a question about the future plans for SQLite. Also about version 4.

Really enjoyed this. Is it possible to subscribe using an email address? The subscribe page seems to ask for a bunch of apps (Apple Podcast etc) that I don't use. Would be great to just receive an email and listen on a browser

If you scroll to the bottom of the page and sign up for the newsletter, I will email you whenever there is a new episode out.

The newsletter will come out once a month when there is a new episode and has a bit of personal take on the episode and maybe some links to other things I've been up to.

If you just want a pure notification, I'm sure you could hook the RSS feed up to an RSS to email service. Thanks for listening!

Great podcast! I have been following it for a while. Can I ask how ended up starting it?

CoRecursive started because I did some episodes as a host on software engineering daily and it was lots of fun so I decided to start one of my own. The first 4 episodes were originally on software engineering daily.

Originally the idea was to give some platform to the areas of computing I was interested in, heavy use of types and functional programming but I think I really found my place as a podcast when I started to focus on peoples stories. So rather than tell me about SQLite, it is tell me about the story behind creating SQLite.

You are so polite, I am impressed with your ability not to push back on the interviewees. In this episode I would have asked "would sqlite really be worse today if you hadn't done your own VCS.". But I think I wouldn't be half as successful as you, people don't wanna be implicitly criticised by their interviewer :P

You are doing so great, favorite podcast by far!

I quite disagree. It's boring when they just say their talking points. The guests are experts in their field, they can handle some pushback and it would be much more educational for the rest of us to hear the answers to the questions we're really wondering about.

It doesn't have to be "implicit criticism", but an open mind doesn't blindly accept stuff, ask for justification.

The justification is where the story is, the tension and relief. This is where you really learn from the interview.

> would sqlite really be worse today if you hadn't done your own VCS.

Yes. Fossil is not only the VCS for SQLite, Fossil is also built around SQLite. SQLite is a core component of Fossil. Thus, when I am working on Fossil, I am forced to interact with SQLite as a "user" instead of as a "developer". In geek-speak, it forces me to "eat my own dog food". This, in turn, prompts me to add needed features to SQLite and more generally to make the SQLite interfaces friendlier to application-developers.

One recent example: SQLite version 3.34.0 added the ability to include two or more recursive terms in a Recursive Common Table Expression. (See item 2 in https://www.sqlite.org/releaselog/3_34_0.html and subsequent links.) This feature was added specifically so that I could more easily write SQL statements that would walk the Fossil version history DAG, as described by the https://www.sqlite.org/lang_with.html#rcex3 link.

I did not develop Fossil with this "dogfooding" idea in mind. It was an unanticipated benefit of Fossil. But in the end, I think it might have been the most important benefit of using Fossil instead of some other VCS.

I'm glad you like the podcast! I do push back when something doesn't sound right. I agree though that I could have asked more about fossil. I really didn't look into fossil that much when I prepared for the interview so it was a bit of a blind spot for me.

Not included in the interview is a discussion we had about types. I really think SQLite should be stricter in enforcement of types and Richard doesn't buy it. I may include it in a future bonus episode but the reason I cut it was it didn't really include any details about story of SQLite, it was just two people disagreeing about types. I've done a lot of episodes about types in the past and I wanted this to be about build SQLite, and not about types.

I was wondering exactly about this. I, for one, would be highly interested in hearing this discussion!

I was wondering why the episodes on the corecursive feed start at number 5.

I'll check your hosted episodes on Software Engineering Daily now!

Thanks, I have interviews on software engineering radio as well: https://www.se-radio.net/team/adam-gordon-bell/

He is on HN. It was a great segment.

Thanks for listening! I'm glad you enjoyed it.

Reply on news.ycombinator.com