Hacker News Reader

Comment by samuelstros | Hacker News Reader

We also came around to dumping files into SQLite instead of dealing with the filesystem (and git) for the lix change control system. This article touches on problems we encountered: https://opral.substack.com/i/150054233/breaking-git-compatib....

- File locking, concurrency, etc., are problems that SQLite solves

- Using SQLite allows drumroll querying files with SQL instead of platform-dependent fs APIs

- Using SQL queries is typesafe with Kysely https://kysely.dev/ (without the need for an ORM)

View on HN

> Using SQL queries is typesafe with Kysely https://kysely.dev/ (without the need for an ORM)

Wow, this is even better than what I've seen people do with F# type providers. Cool cool cool.

the types kysely implemented are crazy. even custom where clause like this one [0] are typesafe :O

[0] https://github.com/opral/monorepo/blob/99356e577f558f4442a95...

Totally random, but I had a guess about the ts error. I hadn't seen kysely before, very cool!

  diff --git a/packages/lix-sdk/src/query-utilities/is-in-simulated-branch.ts b/packages/lix-sdk/src/query-utilities/is-in-simulated-branch.ts
  index 7d677477e..39502f245 100644
  --- a/packages/lix-sdk/src/query-utilities/is-in-simulated-branch.ts
  +++ b/packages/lix-sdk/src/query-utilities/is-in-simulated-branch.ts
  @@ -21,10 +21,9 @@ export function isInSimulatedCurrentBranch(
                // change is  not in a conflict
                eb("change.id", "not in", (subquery) =>
                        subquery.selectFrom("conflict").select("conflict.change_id").unionAll(
  -                             // @ts-expect-error - no idea why
                                subquery
                                        .selectFrom("conflict")
  -                                     .select("conflict.conflicting_change_id"),
  +                                     .select("conflict.conflicting_change_id as change_id"),
                        ),
                ),
                // change is in a conflict that has not been resolved

You are now forever in our git history https://github.com/opral/monorepo/commit/58734e11e51d8e20092.... the ts-expect-error was indeed fixed by your suggestion

Isn’t kind just overriding the type checker? Should that as be necessary?

Yes. The runtime was not affected. Hence, the ts-expect-error. Still nice to have the ts-expect-error fixed :)

In the context of the union query, I think it makes sense. The query takes two different columns (with two different names) from the same table, and combines them together into one column. The TS error happened because the column names didn't match between the two union inputs. But they would never match without the "as".

Cool! I'm surrealize though, sureglymop wrote a sibling comment! I should have just made a PR, lol.

Today i learned that HN has a nesting limit. @surrealize i fixed the username https://github.com/opral/monorepo/commit/7dc1f3c806bd89c6d68...

Haha, thanks!

The conversation in some aspect may have tripped the flamewar detection, which removes the reply button iirc. You can manually reply to a specific comment by clicking/tapping on on its timestamp.

I'm a huge proponent of using SQLite as an abstraction over a filesystem. One warning I will note though, is be aware that a SQLite database does not shrink unless you vacuum it (basically copies the data into a separate file and deletes the original). This is a manual operation you have to do at points where it makes sense within your application. So be careful with disk usage when just writing binary data and then deleting it.

That's a good tip. I was wondering how SQLite avoids page fragmentation. The answer is "it depends," but VACUUM is to the rescue.

What are your reasons for advocating for SQLite as a filesystem abstraction?

You can add as much metadata to the files as your little heart desires.

Regarding your lix project, have you looked at Fossil? It seems like it could maybe do what you're trying to do with some small changes.

Yes. We looked at all version control systems (fossil, pijul, jj, git, darc, sapling). None provide what we need:

- target the browser

- supports any file format, not just text files

- understands changes, not just versioning of files

- suited to build web apps on top

- ultimately enable 1000's of artists, designers, lawyers, civil engineers, etc. to collaborate

we are 2 years into the journey. dec 16, we'll release a public preview

Interesting! Definitely looking forward to that and I hope you succeed.

Reply on news.ycombinator.com