Isomorphic-git v1.0 | Hacker News Reader

Isomorphic-git v1.0

isomorphic-git.org

149 pointskjhughes80 comments

Using this and Gitlab and Netlify you can make static pages that can edit themselves without having a backend anywhere. That is the holy grail of dynamic static. Or Naught Static Pages as I call them :)

I'm not talking about making use of some "serverless" backend like Firebase or something. I'm talking about only using git hosting and static website hosting and in browser computation to make "dynamic" websites.

What you do is:

* You make a regular static page with whatever static web page framework or view template software you like that runs in the browser. I got Handlebars to work. (I'm looking at trying Choo next, because it is literally one hundredth of the size)

* Make another "admin" page where the user can input the data. This page first checks out the current git repo.

* When the user hits submit you use the newly edited data to build the static site in the browser and commit/push everything to git where it is automatically pushed to netlify.

You first have to bootstrap this process on your own machine in the first commit.

I've been holding on to this secret for over a year. Well the cats out of the bag now. I haven't run into someone doing something similar with Isomorphic-git yet. I've had it in production for an artist website for almost a year. I don't know if this is good or bad for Gitlabs or Netlifys business model.

This is a really cool idea. Could you perhaps post a minimal example?

I've been meaning to for over a year :) Maybe this will motivate me.

Really confused about what this does or what advantages it brings.

Dynamic page content without hosting a backend. Backend hosting can be enormously complicated when you just want to be able to edit a few fields in the page dynamically.

How is netlify not the backend

My experience is that the "backend" of a website is where the backend business logic code and the database of the web page is. But in this case it is all running in the frontend in the browser.

Netlify is just the static web page host.

Yeh I guess the logic itself is being executed in the browser to alter state, but the state change is being applied by netlify

Netlify-cms does this, not sure whether it uses isomorphic-git but client side code talks directly to GitHub.

The admin experience leaves a lot to be desired... Slow previews (requires a full build - like a minute!), slow saving, buggy.

Fixing the first two requires a different architecture

It looks like Netlify-cms runs the git and static website software on Netlify backends. So you're not getting the freedom to do whatever you want in the frontend like you do with my approach where the static site generation and git handling happens in the browser. With Netlify-cms you are tied to their implementation of these things on their backend servers.

You can configure netlify-cms to go through netlify for the git operations or not. Going through netlify servers let users edit without github credentials.

On the blog where I have set this up, it goes direct and content editors aren't that happy with the experience

Doing static site generation in the browser would definitely help with previews, at the cost of not working with off-the-shelf static site generators.

I like the "dynamic but static" idea ! Doesn't this already exists if you have a static website but with some dynamic features (js client code calling some serverless functions or querying stuff on AWS for example) ?

I guess here you are adding the "you can edit it from a browser", is that right ?

I was about to make a site just like you describe, having a static website that uses js to dynamically show and edit some content and have it be fetched and stored at some serverless host like Firebase. I guess AWS has something similar to Firebase, but the complexity of AWS keeps me away from checking it out.

There are two benefits to my approach, which motivated me to do it.

1. No backend code or database. Not even serverless. All the complexity of managing this is gone. Don't have to worry about quotas or paying for this either as it doesn't exist. We just need a static site host that can pull and host from git. Netlify does this.

2. Perfect caching performance, as we build a new static website when we change any data. Nothing is generated at view time. The thought of each user calling a backend for the same data over and over on each page load, when the page hasn't changed gives me pain. With this all the data is just built directly into the static site and the static site host and the browser takes care of doing http caching if it isn't changed.

I'm not sure this is a secret, I think I've seen people talk about this in the past. I even did a little bit of work looking into whether or not it was feasible as a replacement for CMS backends like Wordpress, but eventually gave up because:

A) Git operations on both Github and Gitlab are kind of slow (compared to normal AJAX requests).

B) It requires you to embed Git credentials into your webapp clientside, and I'm more comfortable building secure client sessions than I am messing around with libsodium to encrypt credentials locally.

C) The lack of isolation between my data and site code means that I don't have defense in depth if the credentials get compromised. If someone breaks into my session, they can't only compromise my database, they also have push rights to my entire repository. This is especially problematic for apps where I want multiple people to have access, since I often don't trust them to keep their passwords secure.

D) To expand on that, you can't do any validation of input at all -- clientside validation doesn't count. Your only permission you can grant people is "100% commit access to my entire repo, plus (potentially) the ability to execute arbitrary code during my build process", or "nothing".

E) It's also just honestly a lot more complicated than normal hosting. There ended up being a lot of niggly details that I was fiddling around with that weren't terrible to solve, but left me asking, "what exactly is my end-goal here and is it worth the time I'm putting in?"

The security issues mean that having an open submission form is a pretty bad idea anyway, so really you'd only want to do this for a personal site. Except -- I can't think of any scenario where I want to edit a personal website, I have an Internet connection, and I'm on a computer I'm comfortable typing Git credentials into, but I don't have Git installed. Usually it's just going to be easier to make the edit locally and push to Git the normal way.

----

If you are trying to build a multi-user submission form:

You can build a purely static site on Netlify and use 100 hours of 'serverless' functions for free. Your functions can still do stuff like push changes to Gitlab and trigger Gitlab builds if you don't want to fiddle with a database, except now you can actually run real input validation. Plus, if you're using serverless functions, you can make your submission forms work without Javascript, which is a nice plus.

At the point where you're doing more than 100 hours of work a month running builds, you're also probably at the point where the pure-JS repo solution is going to have a really hard time scaling. And if you do reach the point where 100 hours doesn't cut it, you can scale up Netlify functions without rebuilding your entire submission form to use a different architecture.

----

None of that is to say that the setup Sammi describes isn't worth playing around with, or that you can't build a working site with it, or that people should be discouraged from trying. It's very cool; and it's useful to try things like this even just from the perspective that they're fun projects that will make you a better coder.

I just question whether it's really a holy grail, or even a good idea to hook up to your own Gitlab account. I would posit that the reason you see fewer people actually following through and adopting Isomorphic Git in this way isn't because no one's thought of it, it's because you can build the same solution more efficiently and safely for the same price ($0) using serverless functions or (worse-case) setting up a single $5 Linode VPN that you share between all of your sites.

Thanks for the feedback.

You don't need to embed the git credentials in the site. I just made a login form that takes the gitlab credentials. Gitlab gives you 5 users per project, so you can give several people access.

You're absolutely right that this is not a replacement for a full blown CMS, or for when server side input validation is necessary, or for when different people need different edit rights. I've stayed away from using the term CMS for this reason as it implies that complex content management abilities are required. These "dynamic static" pages only give you full edit rights or not, and you need to build everything manually as it is a static website after all.

I have found this approach useful for hosting a static website for an artist where I wanted the artist to be able to easily update some info on the site. I imagine there are a lot of cases like this where you are building a small static website for someone else who isn't a developer and you want them to be able to update the site themselves later, so they don't have to bother you about it.

I wonder how this compares to wasm-git[1], which is libgit2 compiled to wasm for use in both the browser and node. At face value it seems isomorphic-git has been around for longer, and wasm-git seems hot off the press, but the latter seems like maybe it's got more eyes on the prize given libgit2's developer base?

(I have no affiliation with either project, just looking for options to use git in-browser and in node.)

[1]: https://github.com/petersalomonsen/wasm-git

There was a discussion about this two years ago: https://github.com/isomorphic-git/isomorphic-git/issues/268.

Great link, answered all my questions – thank you!

I'd like to comment on the name of this project, but fear my comment would be equivalent to an argument about semantics.

The term has been weirdly adopted by the dev community in a way that kinda makes sense if you squint a little but has always created a dissonance in my brain due to my maths background.

But since we’re here! For those who don’t know the word from outside of the dev sphere, in maths it refers to having 2 way mappings between 2 structures. You can do it between totally seemingly unrelated parts of maths (you just need to show how to map between them). As a result, anything you’ve proved in one space holds true in the other. It can be useful if you have an intuition and understanding of one of the spaces but not the other.

I’m trying to think of a nice example but I haven’t thought about this type of maths for a couple of decades now. Answers on a postcard please!

I remember my mind being blown when in Discrete Math 101 we learned that natural numbers are isomorphic with whole numbers. Was it, you map 0 to 0, even n to n/2, odd n to -(n+1/2) and all of us were like whoa and the TA is like "wait till I tell you about diffrent cardinalities" :D

This is just a bijection - for an isomorphism you need more structure. Roughly, having an isomorphism (which is a function) says:

0. I have two sets of things, and I can perform operations on each set

1. These two things are the same size (this is your example)

2. If I do operation A on elements (a, b) in set A, then perform the isomorphism on the result, this gives me the same thing as if I performed the isomorphism on a and b independently then performed operation B on their result

3. The isomorphism has an inverse with the same properties

[There are various types of isomorphism depending on what part of maths you’re thinking about. But they all have roughly this idea of “same size” and “preserves something cool that I can do in each of the spaces”]

It's an isomorphism in the category of Sets.

(The worst sort of isomorphism)

Yes, I probably ought to have mentioned that. Though ime people are more likely to talk about isomorphisms in other categories, since the only “structure” a set has is its size

How do isomorphic, homomorphic and bijective differ, please?

Isomorphic is homomorphic + bijective.

Bijective means that a mapping is exactly one to one.

Homomorphic means that "structure is kept". E.g. in a vector space you can add and multiply by a scalar, so a mapping f is homomorphic if for all x, y in the vector space and lambda a scalar

    f(lambda*x + y) = lambda*f(x) + f(y)

Short + sweet, thanks

A little correction: an isomorphism is a homomorphism which is bijective and has a homomorphism as its inverse. For many structures the inverse being a homomorphism automatically happens as a logical consequence of the function being a bijective homomorphism, but this isn't the case for all structures. The most prominent examples are topological spaces: a bijection between topological spaces can be continuous (i.e. a homomorphism of topological spaces) but can have an inverse which is not continuous.

The classic example is a function which maps an interval to a circle by applying cos and sin to get the coordinates; this is continuous as small changes in the interval result in a small change in the position on the circumference, but its inverse is not continuous because there will be a point on the circle where the nearby points on one side come from the start of the interval but the nearby points on the other side come from the end of the interval.

What's the meaning of the term in the dev community?

In the JS community, it means a library that can run client-side (in the browser) or server-side (in NodeJS).

Right. While I understand why they use the term this way, it still feels semantically abusive given the math concept.

"Isomorphism" means different things in different communities https://en.wikipedia.org/wiki/Isomorphism_(disambiguation)

Why is the definition in one community special?

Isomorphism means "same shape" from Greek, sounds like a good way to describe code written in the same language running in different environments.

Another term in computing/programming for "same code in different environments" is "portable". (e.g. C++ code that compiles/runs on both Linux/Windows). I've not seen 'isomorphic' used in computing outside of JS.

Not that you're wrong, but it's a deep and pervasive concept in mathematics, and the field of computing is closely connected to mathematics in general. It's sort of a namespace collision.

Ok, but the same arguments could be made about the terms "refactor" and "plane" (as in control-plane/data-plane) but we seem to manage to cope with the overload.

On one hand, you're right. On the other hand, English is basically never consistent, so why should this be an exception?

This isn't an argument about consistency. It's an argument about whether overloaded terms are acceptable. If you want to argue that terms shouldn't be consistent that's an entirely different (and entirely bizarre, IMO) discussion.

Sorry, that was just meant as a dumb joke.

I suspect you might have a personal bias, though.

So, it means cross platform.

Specifically in this context it means that the same JS code can be run in the server and in the browser.

The community has been slowly moving to the term "Universal JS" for that concept, though. Probably partly due to the mathematical ambiguity, but mostly because "universal" is a bit more approachable term.

> The term has been weirdly adopted by the dev community in a way that kinda makes sense if you squint a little but has always created a dissonance in my brain due to my maths background.

I don't really have a 'math background' (engineering graduate, but I don't think that counts, a mathematician certainly wouldn't) but I know (enough, roughly, layman's definition) what it means in mathematics, and work in software engineering.

I have absolutely no idea why this project is called 'isomorphic', it doesn't make any sense to me however much I squint, and doesn't seem to be explained anywhere.

I'd guess it's just an arbitrary name and isn't supposed to have meaning? Though it's a library for `git`, why not just call it `git`? shrugs

The “isomorphic” comes in because the same library / API is available in both the browser and in node. So “anything you can prove in [node] is also proved in [the browser]” (in the context of this library).

I would call that 'portable', personally.

I think more generally it's used to denote code that runs the same on browser and back-end.

"Same" just isn't as fun as saying "Isomorphic"

All the good names were already taken on npm / GitHub: git, git.js, js-git, es-git. "Isomorphic JavaScript" had just been coined and was trendy at the time (2017ish, see https://en.m.wikipedia.org/wiki/Isomorphic_JavaScript) and "isomorphic-git" is unique enough to be Google-able.

Naming things is hard.

This makes me think that there should be two names in package management: a UID, and the import/use-it name.

Many such systems do work like that, but the UID is chosen by the author, so invariably it's a close match to the import/use-it name, and often it's actually the preferred name because the other had to be mangled to be syntactically correct in a program. Consequently, the main display name is the UID; not the import/use-it name.

The UID should be generated by the package manager, and only the import/use-it name chosen by the author. The latter would be the one prominantly displayed, and be allowed to conflict with other packages (that have different UIDs).

Even if you wanted to use two that shared an import/use-it name, there could just be a way of renaming them in the requirements/lockfile, e.g.:

    [dependencies]
    d34db33f = { version = "^0.1", name = "isomorphic_git" }
    l333333t = { version = "^3.2", name = "native_git" }

where usually both of those use the import/use-it name `git`, and the names `isomorphic_git` & `native_git` are whatever I fancied for this project with this dependencies file - could've been `foo` & `bar`.

(Perhaps a more human-friendly format would be `git = "^0.1"`, and we only have to specify something like `= { uid = "d34db33f", ...` when the package manager detects that there is a naming collision. Of course the lockfile, if used, would already have the UID, in case a colliding package was suddenly created.)

I have a similar reaction whenever I come across a library or tool named "manifold".

A simple example would be the isomorphism between a complex number: a + bi and a 2x2 matrix: {{a, -b},{b, a}}.

https://en.wikipedia.org/wiki/Complex_number#Matrix_represen...

From the project: "The "isomorphic" in isomorphic-git means that the same code runs in either the server or the browser. That's tricky to do since git uses the file system and makes HTTP requests. Browsers don't have an fs module. And node and browsers have different APIs for making HTTP requests!"

A thread from 2018: https://news.ycombinator.com/item?id=17083807

It’s fun to see everyone in 2018 was arguing about Rust too

I can see how having git in javascript makes workflows with git in js based tools like webpack and rollup much more convenient to install and share.

I'm having a failure of imagination on the browser side though - anybody got killer use cases running git in the browser?

I am personally using Isomorphic Git exclusively in my automation scripts (written in Node) where it is about infinity times more convenient than shelling out to Git and parsing its output (something which feels dirty to me even in scenarios where it works, like in VS Code's version control pane which does this), however I know a few folks are looking at online IDEs and various things with P2P - like using WebRTC to mesh up clients and using Git in a truly distributed fashion across this WebRTC mesh. The fact that you can supply your own fs implementation makes these kind of applications really easy.

An example would be Glitch or Codepen creating commits on each code change, to allow undo/redo of a project over a broader range of time.

Glitch already has this feature although it’s currently a bit buggy. It may even be based on isomorphic git, not sure.

Further - I am using git commits to store hashes for code-created artworks, so that I can reproduce and remix the exported PNG at a later point. See here:

https://github.com/mattdesl/canvas-sketch/blob/master/docs/e...

Client side git would mean one less reliance on nodejs.

git backed personal wiki sites exist, though I don't know what they generally use. e.g. TiddlyWiki

I'm wondering if I can use this in an ERP system I'm developing. A feature request has been to see a diff history of actions on an entity (say, Purchase Orders). Using something like this, I believe I could make a 'commit' on each save, then display a history of commits to the user to 'diff' against to see previous revisions (who added which items, when did the PO number change, etc).

I was planning on storing this history in the database as raw JSON, then diffing the JSON server-side, but this may provide a more robust solution.

That could work, but note that there are other libraries that are purpose-built to do diffing without the rest of git's many features. We are using jsondiffpatch [1] and quite happy with it. It also has builtin formatters to output e.g. html view of the changes.

[1] : https://github.com/benjamine/jsondiffpatch/blob/master/READM...

Oh that is sweet! Thanks for the tip!

Let say you are building a document editor web app, or any app for creating content. Instead of simple undo/redo you could leverage git powers for versioning + tagging + branching + collaboration on document content. I needed this kind of functionality on one project, but I haven't yet looked into isomorphic-git to see if it is actually a good fit for such purpose.

Pretty much this. I’m building a WYSIWYG note taking app that syncs with GitHub, sort of like Prose.io but more focused on the note taking than CMS part (although the option of publishing notes would combine these two).

I think there’s two benefits of isomorphic git for this use case.

1. It’s more api agnostic: if I wanted to support Gitlab, I wouldn’t have to write substantially more code tailored specifically for Gitlab’s API because it speaks git. You would only have to switch out Github's auth for Gitlab's auth. I also found that add multiple files and versioning them with Github’s API is a pain; they’re remaking a client called Octokit but it’s pretty underdeveloped right now.

2. Offline version control: you shouldn’t have to be online to take notes. With isomorphic git I can add and commit, and then push back to GitHub later when I have Internet. Fun fact: to get git to work on the browser, the author of isomorphic-git wrote a library to port NodeFS commands backed by IndexedDb. This means you can also have offline image drag-and-drop.

There are some minor issues with isomorphic-git when I checked a month ago.

- Largest one is only fast-forward merges are supported; not sure how to get around this one besides cloning the remote Github branch and maybe manually merging files.

- Another one is a bug with git init followed by git add remote and git pull which leads to a reference error because no local commit has been made yet; you can imagine a user who immediately logs in with Github so you want to sync with a pre-existing repo.

[1] https://github.com/isomorphic-git/lightning-fs

Ok - but isn't the hard part of collaborative authoring the conflict resolution not the storing of versions per se. Git has a merge mechanism, but it's not necessarily that user friendly?

Isn't something like OT ( operational transforms ), or CDRT ( Commutative Replicated Data Type ) a better way to do this?

Conflict resolution of course depends on actual content. Source code could be among most tricky ones to auto-resolve or detect when manual merge is necessary. Documentation, art and other not-so-strict content could be more auto-resolvable than source code. Git is here simply a mechanism for allowing different work flows. Your web app should enforce best practices for non-expert users. The added benefit is developer familiarity with git concepts.

I wasn't familiar with OT and CDRT, so maybe I'm off the mark. They seem to try to solve real-time collaboration between multiple users editing same part of same document. This would always require some meta-document or meta-channel where users coordinate what should be changed, in what way and who does what. I would argue that git model is better: user decides when to pull and when to push changes, and the communication between collaborators is outside the scope of the versioning system.

Collaboration is just one application. Even single-user content creation could vastly benefit from built-in versioning. This would eliminate some mental burden and usage friction (tagging), and open up more possibilities to experiment with different directions (branching). The feeling you get when you check-in working version of your code and you are now free to break stuff, that kind of feeling is missing when creating slides, graphics, music, CAD design...

Fork-able web content (e.g. blog posts, class notes) is something I've been looking at, but you would still need to rely on an online service like Github or Gitlab.

I’d love to see some work put in to making this work with React Native. I spent a day trying to get this to work a few weeks ago with no joy, and I couldn’t find any indication that anyone else had managed to make it work either.

This would seem like a workaround but it's possible to run node.js on Android and iOS and bundle it with your react native app so that might be a potential way forward: https://code.janeasystems.com/nodejs-mobile/

I tried to run it in RN year ago with no luck, filled an issue [1]. Looks like they removed Buffer dependency in 1.0 which was one of the problems.

[1] https://github.com/isomorphic-git/isomorphic-git/issues/698

edit: fix link

I think you accidentally pasted a link to wasm-git instead of the Isogit issue link.

Thank you! fixed

But why? Aren't your users gonna be far better off if you used native git?

Sorry, I know this comes off as a bit dickish, but I'm interested to know the circumstances that lead to this point of view :)

More or less same arguments as for choosing React Native for any other project: share some or all code on web/android/ios platforms.

The other option could be compiling libgit2 to native code/wasm and use it in RN and web, but it will be harder then just import js-only npm.

Depends what users, if you're targeting some education format or institution that wants some locking down of their OS git native is kind of a non starter. No one can configure it for you, there's 65 million install options and you're loading a bash shell into windows.

Very different from "this app has packaged tutorials and tools just start at lesson 1"

My particular use-case is that I'm not a Swift/Objective-C developer, I'm a JavaScript developer.

React Native opened the door to allow me to build apps and, realistically, I'm not going to spend the time battling through trying to get native git working through React Native. If there's a native developer out there that wants to tackle this problem and wrap it up in a nice JS module then I'd be in your debt!

Great library. Wish it worked with local urls tho.

I love this library! ;)

please devs stop using big words you don't understand

Reply on news.ycombinator.com