yes. unfortunate. we choose the name before they launched. by the time they launched, lix (vcs) was already used in prod
555 karma · joined December 3, 2021
The appeal or structured file formats like .docx, .json, etc. Images are unstructured and simple "do you want to keep the left or right image" is good enough.
Take a docx, write the file, parse it into entities e.g. paragraph, table, etc. and track changes on those entities instead of the binary blob. You can apply the same logic to files used in game development.
The hard part is making this fast enough. But I am working on this with lix [0].
I built lix [0] which stores AST’s instead of blobs.
Direct AST writing works for apps that are “ast aware”. And I can confirm, it works great.
But, the all software just writes bytes atm.
The binary -> parse -> diff is too slow.
The parse and diff step need to get out of the hot path. That semi defeats the idea of a VCS that stores ASTs though.
We were coming from a an application perspective where blocking the users intent is a no-go.
Do you have a link to a discussion where the JJ community is discussing checks?
Begs the question if Anthropic will follow up with a first-class Claude Code "multi agent" (git worktree) app themselves.
Write an SSIS plugin (essentially just parse the XML) and lix can track the xml props in the SSIS package. Opposed to line by line stuff that git does.
Hm that is what lix provides?
SQL is just the interface to query the deltas.
Anyhow, lesson learned. The primary use case for lix is embedding. Positioning lix for binary files leading to existing systems. None of those support the embedded use case. Thus, don't position lix for binary files :)
> How often are binary files being diffed? How long does it take to materialize? How long to run a diff algorithm?
If version control is embedded in an app, constantly.
Imagine a cell in a spreadsheet. An application wants to display a "blame" for a cell C43 i.e. how did the cell change over time?
The lix way is this SQL query
SELECT * from state_history WHERE file_id <the_spreadsheet> AND schema_key "excel_cell" AND entity_id C43;
Diffing on the fly is not possible. The information on what changed needs to be available without diffing. Otherwise, diffing an entire spreadsheet file for every commit on how cell C43 changed takes ages.
The way to write a plugin:
Take an off the shelf parser for pdf, docx, etc. and write a lix plugin. The moment a plugin parses a binary file into structured data, lix can handle the version control stuff.
Lix uses SQL databases as storage and query engine. Aka you get a filesystem on top of your SQL database that is version controlled.
Or, the analogy to git: Git uses the computers filesystem as storage layer. Lix uses SQL database (with the nice benefit of making everything queryable via SQL).
> Lix doesn't reinvent databases — durability, ACID, and corruption recovery are handled by battle-tested SQL databases.
>> This seems like a left turn from the value prop and why the value prop matters?
Better wording might be "Lix uses a SQL database as storage layer"?
The SQL part is crucial for two reasons. First, the hard part like data loss guarantees, transactions, etc. are taking care of by the database. We don't have to build custom stuff. Which secondly, reduces the risk for adapters that data loss can occur with lix.
> As for how it works, both in the docs and in the comment I'm replying to, it's unclear how any of this interacts with the native version control embedded in M365 apps or why this tool can be trusted as effective at tracking M365 content changes.
It doesn't interact with version control in M365.
I'll update the positioning. Lix is a library to embed version control in whatever developers are building. Right now, lis is mostly interesting for startups that build AI-first solutions. They run into the problem "how do customers verify the changes AI agents make?".
The angle of universal version control, and using docx or Excel as an example, triggers the wrong comparisons. By no means is Lix competing with Sharepoint or existing version control solutions for MS Office.
In case of a conflict, you can either decide to do last write wins or surface the conflict to the user e.g. "Do you want to keep version A or version B?"
The SQL engine is merge unrelated. Lix uses SQL as storage and query engine, but not for merges.
AI agents are the pull right now to why version control is needed outside of software engineering.
The mistake in the blog post is triggering comparisons to git, which leads to “why is this better/different than git?”.
If you have a custom binary file, you can write a plugin for it! :)
Lix runs on top of a SQL database because we initially built lix on top of git but needed:
- database semantics (transactions, acid, etc.)
- SQL to express history queries (diffing arbitrary file formats cant be solved with a simple diff() API)
But, Lix use case is not version controlling code.
It’s embedding version control in applications. Hence, the reason why lix runs within SQL databases. Apps have databases. Lix runs of top of them.
The benefit for the developer is a version control system within their database, and exposing version control to users.
There is nuance between git line by line diffing and what lix does.
For text diffing it holds true that diffing is a separate layer. Text files are small in size which allows on the fly diffing (that's what git does) by comparing two docs.
On the fly diffing doesn't work for structured file formats like xlsx, fig, dwg etc. It's too expensive. Both in terms of materializing two files at specific commits, and then diffing these two files.
What lix does under the hood is tracking individual changes, _which allows rendering a diff without on the fly diffing_. So lix is kind of responsible for the diffs but only in the sense that it provides a SQL API to query changes between two states. How the diff is rendered is up to the application.
Lix is not a replacement for git. Nor does it target version controlling code as the primary use case.
A better positioning might be "version control system as a library". The primary use case is embedding lix into applications, AI agents, etc. that need version control.
I need to to bed now. I have a flight to catch in 6 hours.
PS I am open to suggestions regarding the positioning!
File change -> Plugin (detects changes) -> Lix
It works surprisingly well because most standard file formats have off the shelf parsers. Parse a file format, and et voila, it is trivial to diff. Then pass on a standard schema for changes to lix and you end up with a generic API to query changes.
I'm the creator of lix.
Lix doesn't target code version control. It can be used for it. But the primary use case is embedding version control in applications. Such an application can be an AI agent that modifies files which entails the need to show what the agent did in that file e.g. tracking the changes.
Git is good enough for code. I don't think there is space to gain much market share.
Someone can write a CLI for it. Though, the primary use case is not code version control but embedding into applications
In general, lix gives in API to track changes in any file format (via plugins). The "diff noise" thus depends on a) the plugin i.e. does it track them metadata? and b) what is rendered as the diff.
If the user doesn't care about seeing a diff of metadata in Excel, don't render the metadata in the diff. The latter is trivial because diffing in lix is just a SQL query.
Going through the questions now. So much for going to bed.
That's what I meant with overwhelming / too niche.
It seems like you intend to productize the RFC process e2e. But most "time consuming" parts of an RFC process is the human stuff "Did you read this?" "Did you update the RFC again?" etc. That back-and-forth seems to be expressed by all the features you have in RFC Hub but:
1. That makes RFC hub complicated.
2. Requires buy-in from every party to participate in all of RFC hubs feature like "Yes, I reviewed it and pressed the reviewed button in RFC Hub"
1 & 2 combined make RFC Hub (likely) a very niche product. New users are overwhelmed. Existing users need to onboard new users (their collegues) though. Otherwise, the RFC process will fallback to just DMs on Slack. Only a few teams will have sufficient buy in from all team members.
GDocs might be annoying to track who read the RFC etc. etc. but everyone is familiar with it.
I write RFCs, I share RFCs and your tool seems to require a substantial amount of buy in
- register
- unclear what the writing experience is
- outdated / overloaded UI
The last RFC I wrote was in hackmd (https://hackmd.io/Jjy-afCWS4CAFlHa62anMQ) because
- I wanted Markdown to store the RFC in git eventually
- Google Docs has issues with Markdown rouundtrips
- I didn't want to use git to write with VSCode (although... I actually did. I let CLaude Code write most of the RFC under my guidance, then put it into hackmd for easy sharing)
I hope the feedback helps!
Take a look at Monitor Control how they structured their GitHub Readme and release page to download a dmg https://github.com/MonitorControl/MonitorControl
for context, i want to build a claude code like agent in a WYSIWYG markdown app. that's how i stumbled on your blog post :)