brendoncarroll

No Nonsense Containers: a simple Linux containerization tool, secure by default, with support for presets.

https://github.com/brendoncarroll/nnc

I use it for running agents locally.

  nnc run /usr/bin/opencode --preset agent

You can make your own presets (which nnc looks for in ~/.config/nnc/presets) or use one from the standard library

https://github.com/brendoncarroll/nnc/tree/master/presets

Presets are written in Jsonnet, and resolve to a list of things to pass through from the parent process into the container. Presets can reference other presets, so you can build up arbitrary rules for passing files and devices into containers, give those rules a name, and reference them later.

Thanks for the link.

It looks like this treats files as blobs just like Git, and trees as single objects which fit in memory. Assuming that is a correct understanding, this core abstraction would need to change to handle large files and directories well.

All the well known version control systems do this though, and it simplifies the system significantly. It's the right model for source code, but it doesn't translate well to arbitrary data.

I've heard that jj has support for non-git backends? Can anyone comment on how difficult it would be to add support for another backend, any docs or examples?

I have a project[0] that does the large file thing well, but is missing most of the version control porcelain. I've been looking for the path of least resistance to integrate it into something with a larger user base.

[0] https://github.com/gotvc/got

I'm working on Got, version control like Git, but for files and directories of any size and E2E encrypted.

https://github.com/gotvc/got

Got is built on Blobcache, which is a general-purpose transactional storage layer and E2E encrypted backend.

https://blobcache.io

https://github.com/blobcache/blobcache

Yes the author reached out. There has not yet been a confusion among real users that I am aware of.

https://github.com/gotvc/got/issues/20

> The problem is when you move beyond text files it gets hard to tell what changes between two versions without opening both versions in whatever program they come from and comparing.

Yeah, totally agree. Got has not solved conflict resolution for arbitrary files. However, we can tell the user where the files differ, and that the file has changed.

There is still value in being able to import files and directories of arbitrary sizes, and having the data encrypted. This is the necessary infrastructure to be able to do distributed version control on large amounts of private data. You can't do that easily with Git. It's very clunky even with remote helpers and LFS.

I talk about that in the Why Got? section of the docs.

https://github.com/gotvc/got/blob/master/doc/1.1_Why_Got.md

Me too. Version control is great, it should get more use outside of software.

https://github.com/gotvc/got

Notable differences: E2E encryption, parallel imports (Got will light up all your cores), and a data structure that supports large files and directories.

I also wrote a tool for doing this[0], after one of these agents edited a config file outside of the repo it was supposed to work within. I only realized the edit because I have my dotfiles symlinked to a git repository, and git status showed it when I was committing another change. It's likely that the agents are making changes that I (and others) are not aware of because there is no easy way to detect them.

The approach I started taking is mounting the directory, that I want the agent to work on, into a container. I use `/_` as the working directory, and have built up some practices around that convention; that's the only directory that I want it to make changes to. I also mount any config it might need as read-only.

The standard tools like claude code, goose, charm, whatever else, should really spawn the agent (or MCP server?) in another process in a container, and pipe context in and out over stdin/stdout. I want a tool for managing agents, and I want each agent to be its own process, in its own container. But just locking up the whole mess seems to work for now.

I see some people in the other comments iterating on what the precise arguments to bubblewrap should be. nnc lets you write presets in Jsonnet, and then refer them by name on the command line, so you can version and share the set of resources that you give to an agent or subprocess.

[0] https://github.com/brendoncarroll/nnc

I personally think that this is the future, especially since such an architecture allows for E2E encryption of the entire database. The protocol should just be a transaction layer for coordinating changes of opaque blobs.

All of the complexity lives on the client. That makes a lot of sense for a package manager because it's something lots of people want to run, but no one really wants to host.

I recently released Blobcache v0.0.2. https://github.com/blobcache/blobcache

Blobcache is a content-addressed data store for holding application state, and buiding E2EE applications. This most recent release includes a git remote so you can push and fetch Git data into and out of Blobcache.

I'm a happy bcachefs user. Haven't had any issues on a simple mirrored array, which I've been running since before it was in (and out) of the kernel. It's the best filesystem in 2025. Thank you for all your work.

What is the status of scrub? Are there any technical barriers to implementing it, or is it just prioritization at this point? FWIW I think there are probably a lot of sysadmin types who would move over to bcachefs if scrub was implemented. I know there are other cooler features like RS and send/receive, but those probably aren't blocking many from switching over.

I work on a project Blobcache, a content addressed store for exposing and consuming storage over the network. It supports full end to end encryption, and offers a minimal API to prevent applications from leaking data.

https://github.com/blobcache/blobcache/blob/master/doc/0.2_W...

You can persist arbitrary hash-linked data structures in Blobcache volumes. One such data structure is the Git-Like Filesystem, which supports the usual files and trees.

https://github.com/blobcache/blobcache/blob/master/doc/8.5_G...

The same API part isn't surprising, content addressed stores are the most natural way to accept encrypted data.

The public storage networks are targeting a different use case than Blobcache though, which I think of as a private or web-of-trust storage network. To use a cryptocurrency backed storage solution, one must manage accounts, or a wallet of transaction outputs, connect to unknown parties on the internet, and pay for the increased redundancy. There's also legal risk, depending on the jurisdiction, when allowing untrusted parties to store arbitrary information on one's devices.

I don't want to consult the global economy in order to make use of my extra hard drives, which would otherwise be idle.

I couldn't find an email in your bio. You can reach me via the email at the bottom of my website (in my HN bio).

Looking through the docs on Peergos, it looks like it's built on top of IPFS. I've been meaning to write some documentation for Blobcache comparing it to IPFS. I can give a quick gist here.

Blobcache Volumes are similar to an IPNS name, and the set of IPFS blocks that can be transitively reached from it. A significant difference is that Blobcache Volumes expose a transaction API with serializable isolation semantics. IPFS provides distributed, available-but-inconsistent, cryptographically signed cells. IPFS chooses availability, and Blobcache chooses consistency. A Blobcache Volume corresponds to a specific entity maintained and controlled by a specific Node. An IPFS name exists as a distributed entity on the network.

Most applications need some sort of consistent transactional cell (even if they don't realize it), but in order to be useful, inconsistent-but-available cells have to be used carefully in an application specific way. I blame this required application-specific care for the lack of adoption of CRDTs.

There's a long tail of other differences too. IPFS was pretty badly behaved the last time I used it, trying to configure my router, and creating lots of connections to other nodes. Blobcache is more like a web browser; it creates transient connections in immediate response to user actions.

That whole ecosystem is filled with complicated abstractions. Just as an example, the Multihash format is pervasive. It amounts to a tag for the algorithm used to create a hash, and then the hash output. I'd rather not have that indirection. All the hashes in Blobcache are 256 bits, and you set the algorithm per Volume. In Go that means the hashes can just be `[32]byte` instead of a slice and a tag and a table of algorithms.

I haven't used IPFS in a while, but I became pretty familiar with it awhile ago. Had I been able to build any of the stuff I was interested in on top of it, I probably wouldn't have written Blobcache.

I work on a FOSS project in this space, Blobcache.

https://github.com/blobcache/blobcache

Trusting a server to store an application's state is a different thing from trusting it to author changes or to read the data. Servers should become dumber, and clients should become smarter. When I use an app, I want the app to load E2E encrypted state from storage (possibly on another machine, possibly not owned by me) make whatever changes and produce new encrypted data to send back to the server. The server should just be trusted for durability, and to prevent unauthorized access, but not to tell the truth about doing either of those things. Blobcache provides an API to facilitate transactions on E2EE state between a dumb storage server and any smart client.

Blobcache can be installed on old hardware along with a VPN like Tailscale and then loaded up with data from other devices. Configuration is like SSH, drop a key in a configuration file to grant access. It removes most of the friction associated with consuming and producing storage as a resource.

I'm using it to build E2EE version control like Git, but for your whole home directory.

https://github.com/gotvc/got

I'm working on Blobcache. https://github.com/blobcache/blobcache

Blobcache is content addressed storage, available over the network. Blobcache allow nodes to securely produce and consume storage. Configuration in similar to SSH, drop a public key in the configuration, and you're done. Blobcache is a universal backend for E2E encrypted applications.

Docs - https://github.com/blobcache/blobcache/blob/master/doc/0.0_B...

I'm also working on Got Version Control https://github.com/gotvc/got

Got uses Blobcache for storing file data.

Got is like Git, if you fixed all the problems with storing large files and directories in Git. There's no "large files provider" to configure separately. All the data for a commit goes to the same place. Got also encrypts all the data you put in it, E2E. If you've run into problems putting your whole home directory in Git, you might have more luck with Got.

Both projects are GPL licensed, FOSS. Contributions welcome.

All of those issues can be solved by doing an import of the changed file into the build system's content addressed store, and creating a new version of the entire input tree. You also don't need to choose between cancelling, waiting, or dropping. You can do 2 builds simultaneously, and anything consuming results can show the user the first one until a more recent one is available. If the builds are at all similar, then the similar components can be deduplicated at runtime.

These techniques are used in a build system that I work on[0]. Although it does not do automatic rebuilds like Poltergeist.

[0] https://github.com/wantbuild/want

I'm not sure which paradoxes you are referring to. Type systems are used for a lot of things, in Mycelium a Type is an encoding strategy for it's Values. And just like I could explain the encoding strategy to you in text, the Type can be stored as bits representing that strategy, so a machine can read the Type and know how to decode Values using the strategy. Eventually this ends with predefined constants at the Type of a Type of a Value level, so there's a fixed point instead of an infinity.

The serialization format solves a similar problem to Protocol Buffers or JSON. If you haven't heard of either of those, then Mycelium might not solve a problem that you care about. Just after your quote the README mentions things like Products and Lists which both Protocol Buffers and JSON have support for in the form of Messages/Repeated and Objects/Lists respectively.

Mycelium has some interesting design choices compared to JSON and Protocol Buffers. Everything is built up from Bits, there is a Bit type which contains the values 0 and 1. Bytes are `Array[Bit, 8]` and Strings are `List[Byte]`. A 32 bit integer would be `Array[Bit, 32]`. There are also Sum (Coproduct) types, and cryptographic pointer types (called Refs in Mycelium).

Mycelium can be used to solve the same problems as those technologies. That's sort of table stakes for a serialization format. Mycelium additionally tackles the problem of sending procedures (called Lambdas in Mycelium) over the wire as well. That is a fairly simple feature to explain (get my procedure from here to there, it works with strings why not functions?), but it implies a significant amount of technology including a machine code specification and abstract machine model to execute it.

As for practical applications. Mycelium is suitable to be used as:

- A serialization format for storage and transfer.

- A VM with well controlled access to external resources for applications to run untrusted code.

- The VM can be a compiler target for programming languages. (Spore https://github.com/mycweb/mycelium/tree/master/spore is one such language)

- A format for data structures which need to be cryptographically signed. All Mycelium data structures are Merkle Trees.

- Large data structures which need to be efficiently synced. All Mycelium Values can be synced efficiently by traversing the cryptographic pointers and skipping values which are already available locally.

The Want Build System

https://github.com/wantbuild/want

https://doc.wantbuild.io

Want is a hermetic build system, configured with Jsonnet. In Want, build steps are functions from filesystem snapshots, to filesystem snapshots. Want calls these immutable snapshots Trees. Build steps range from simple filesystem manipulation and filtering, to WebAssembly and AMD64 VMs.

I don't think you need to go quite so far as checking gigabytes of executables into version control. If you download some dependencies at build time, that's fine as long as you know exactly what they are ahead of time. "Exactly what they are" means a hash, not a name and version tag.

The dockerized build approach is actually a good strategy, unfortunately it's done by image name instead of image hash in practice.

Upgrading dependencies, or otherwise resolving a name and version to a hash is a pre-source task, not a from-source task. Maybe it can be automated, and a bot can generate pull requests to bump the versions, but that happens as a proposed change to the source, not in a from-source task like build, test, publish, or deploy.

I'll throw in INET256 https://github.com/inet256

It's a specification for identity based networking. There is a meshnet and a centralized implementation. You can layer IPv6, IPv4, or application traffic on top of any compatible implementation.

> no need to exchange the key during authentication

The public key does need to be exchanged, along with a signature relating it to the current session. This is all handled by TLS, there is no need for the client to send the key in the application data.

> You still need to generate it and distribute it to the user

This approach avoids distributing secret key material at all. Private keys should ideally never move. They are generated randomly, used to derive the corresponding public key, and then persisted as appropriate. The public key is sent around to other parties.

It's amazing how far adrift the industry has gone with authentication.

This post is saying to avoid OAuth and use bearer tokens because OAuth is too complicated. I agree with OAuth being too complicated, but I don't really think bearer tokens are the solution either.

Now there are jwts, and passkeys, and all these other solutions, which from where I'm standing, just look like someone else's resume-builder that I'm going to have to understand in a few years in order to do something simple and unrelated.

We are already connecting over TLS, just have the client authenticate with a long lived asymmetric key. Let me see that key in whatever web framework; I should have access to it, same as an HTTP header. Then I'll stick it in the database, maybe hash it first, if it's big. It doesn't have to be harder than that, your identity is (the hash of) your public key.

My approach to hosting with Got has been to make it easy and secure for users to host from any machine.

INET256 solves that problem nicely. If you have access to an INET256 network, then all you have to do is swap addresses and two Got instances can communicate.

https://github.com/inet256/inet256

Also, end-to-end encryption is table stakes. Any data that leaves the user needs to be encrypted in transit, and if it hangs around away from the user, at rest.

Yep, your link is indeed to the probabilistic tree used in GotKV.

Here "probabilistic" just refers to a way of balancing a tree. Rather than having a set of rules to keep the tree balanced, like with a btree or red-black tree, balancing decisions are made pseudorandomly. The result is that the tree is very likely to be balanced, and is unlikely to be unbalanced.

In the case of GotKV's tree: the entries are stored together in a stream, and for each entry a hash is computed. If that hash is lower than a certain value then the entry is considered a split point, and a tree node is created. So now we have a stream of entries, divided probabilistically into sections. Each section is a tree node. Now take references to those nodes and turn them into entries, and repeat the process, so you have fewer nodes. That continues until you have one node, which is the root. This technique is very similar to content defined chunking, and some probabilistic trees are implemented using content defined chunking on their record format, rather than a pseudorandom value calculated per entry, as in GotKV.

For those unfamiliar with probabilistic data structures, I highly recommend trying to understand skip-lists first. At least why they are balanced.

https://en.wikipedia.org/wiki/Skip_list

As an aside, one of the neat things about GotKV is that the keys are delta-encoded. Adding or removing a prefix from every key in the tree is a constant time operation. This might be obvious to some of the database folks out there, but it's a fun mind-blower if you haven't encountered the technique before.

Sure, Git stores data in a trie. Each file is one blob identified by hash, and directories (called trees in Git) are blobs where each line is a directory entry with a name and the hash of a file or another tree. This means that modifying an object /down/a/long/path/like/this.txt has to create copies of all the trees on the way up. The technical term for this is "write amplification", and in Git it is affected by path length among other things.

Got stores data in a probabilistic tree (GotKV[0]). The number of nodes before you get to data will scale logarithmically with the size of the entire filesystem, not the depth of a specific object.

Then there is the issue of large files. A file in Git is always 1 blob. Syncing a large blob is not easy because if you are interrupted and have to restart, you have lost all your progress. You can't verify the hash of a blob until you have the whole thing. Got has a maximum blob size, so you'll only be buffering <2MB at a time before you can verify that the blob is correct. If a transfer is interrupted, the most you'll have to repeat is one blobs worth, plus any tree nodes above that blob.

Compared to rsync, Got uses variable size chunks and a faster content defined chunking algorithm, recently featured here on HN[1]. I haven't thought about if variable vs fixed chunks is better for file transfer, but for version control, the higher chance of convergence is important. It means you have better deduplication.

[0] https://github.com/gotvc/got/tree/master/pkg/gotkv

[1] https://news.ycombinator.com/item?id=34303497

This is a very similar to one of my projects "Got".

https://github.com/gotvc/got

The algorithms it uses are superior to rsync and git in a few ways. It comes short on features, especially for software development compared to Git. The motivation is more for personal file storage.

I notice you're using Go and AGPL licensed, so you could borrow any of Got's libraries without issue. (Got is GPL licensed.) Definitely reach out in a GitHub issue.

I̶ ̶d̶o̶n̶'̶t̶ ̶t̶h̶i̶n̶k̶ ̶r̶s̶y̶n̶c̶ ̶u̶s̶e̶s̶ ̶f̶i̶x̶e̶d̶ ̶s̶i̶z̶e̶d̶ ̶c̶h̶u̶n̶k̶s̶.̶ The algorithm is described here; it's a rolling hash.

https://rsync.samba.org/tech_report/node3.html

Your description of content defined chunking is exactly right though. There are a number of techniques for doing it. FastCDC is one of them, although not the one used in rsync.

https://en.wikipedia.org/wiki/Rolling_hash

EDIT: Corrected in the comments below. Fixed sized chunks searched for at any offset with a rolling hash. The rsync algorithm description is here.

https://rsync.samba.org/tech_report/node2.html