Making GitHub More Open: Git-backed Wikis | Hacker News Reader

Making GitHub More Open: Git-backed Wikis

github.com

145 pointstechnoweenie29 comments

Awesome! The wiki was the weakest part of the github ecosystem, and it's wonderful to see it get some much-needed love. Besides being git-backed (and therefore portable), the new wikis also finally have markdown support. The pace of releasing new features that Github is able to maintain is incredible.

Now - if we can get Issues doing the same thing by building on top of something like YABT or some other text-based approach to issue tracking, that'll be awesome.

Definitely. Not having offline access to issues is the only reason I don't use GitHub as my sole issue tracker.

Where's the Readme? It's hard to tell if the YABT approach has any merit without any documentation.

I'll try to address that over the weekend. YABT was my original foray into Python, so it's pretty week right now and never really developed into much more than a proof of concept.

Yeah, or Emacs's org-mode format. That'd work very well as a machine readable text-based bug and tasklist format.

That would rock. Scott hacked on http://github.com/schacon/ticgit for a while. I'm not totally sure what happened to that. We're all in favor of a git-backed issue tracker. Though, I think it offers some challenges in filtering, labeling, etc from a git repo.

YABT popped up shortly after ticgit -- same type of thing. Scratched and itch to try something, then let it fall off. I'm going to revisit it and see where it stands though. I'd love to get it to where it serves as the backend for GitHub Issues.

One of the reasons I wrote it instead of extending ticgit was that I wanted something entirely independent of any VCS. All stuff I need to get in a readme, though. :-)

I worked on ticgit a little too. It's such a neat idea, but I'm not sure how it would work in practice. I think git repos would be best for syncing, but not the main data store on GitHub.

I'm curious about using CouchDB, actually. It has similar versioning and replication features to git.

Have you looked at ditz?

http://ditz.rubyforge.org/

Yup, I tried using it and ticgit for a little while. Didn't really care for either of them.

Yup - I tried a half dozen while thinking about starting YABT. None of them seemed to fit the bill just right.

If the storage API inside the project is good enough (something YABT's isn't yet), swapping out where it gets the data is a minor issue. You could use Git for the main storage, then have a web frontend that reads from Couch/Mongo/Redis/NewHotnessNotYetDiscovered and lazy loads from disk. A post-receive hook or two for expiring data in that cache and you're set.

+1 for distributed issue tracking. I'm a BitBucket user, where the wiki has been Mercurial-backed for some time now, but I'd like both sites to go distributed on their issue tracking. Then you'd get complete portability on any project.

I looked at Fossil for this reason (distibuted src/wiki/issues), but wasn't convinced that it was a real Mercurial or Git replacement for source control, so I gave up for now. Maybe something like Bugs Everywhere will do the trick.

Sounds like a good idea. Because, why should your wiki use its own custom version control system?

The other question you need to ask: how good of a database is your version control system? Git definitely provided some challenges in this regard. One example: we can't simply list recently updated pages. We keep them in a Redis sorted set :)

UPDATE: Okay, Scott Chacon just popped into Campfire to school me in git: `git diff-tree --name-only HEAD~10 HEAD`

One reason is scalability and distributability, though that doesn't apply to smaller wikis. There have been attempts to load Wikipedia's full history dump into a git repository, but git doesn't seem to be able to handle that volume of stuff (and frequency of edits) nearly as well as Wikipedia's current setup of distributed MySQL servers and custom code does.

Though you make valid points, the scale of a typical Github wiki is nowhere near the scale of Wikipedia.

Yeah, and the biggest edit bottleneck---git's serialization of updates---is probably not an issue either, since it only applies to wikis with a lot of pages that get a lot of edits.

(By serialization of updates, I mean that, afaik, git can't apply two patches simultaneously, even if they're to completely disjoint sets of files; it has to apply one patch, then the other. That'd be like using a db that requires you to lock the whole db to do any writes. High-volume wikis instead will only lock the rows for the particular page they're editing, so edits to other pages can happen simultaneously without blocking.)

Wikipedia has zero branching and doesn't require atomic commits across a range of articles. It's a mismatch for SCM.

Wikipedia has branching now in the form of "approved revisions", and it has a lot of long standing TODOs like annotate that would be solved by a SCM.

The storage backend for articles is its own mini-scm with a revision table + text storage. There's no reason why a Git based backend would be any worse of a fit for a wiki.

Are approved revisions really a branch or just a tag somewhere in a linear version history? With branches you can have n parallel branches, whereas with Wikipedia it seems--though I may be wrong--that you just have one version history, with one or more revisions within that history flagged as "approved". (Having n branches of a Wikipedia article would be frowned upon, since the community decided in the early days that you can't fork an article--that only gets in the way of building an authoritative and neutral version--though it has been argued in the past that having multiple forks on controversial topics, each written from a particular point of view, would be more informative.)

Also, Git-style branches only make sense as alternative branches of the entire repository, which doesn't exist in Wikipedia terms because you can't atomically commit changes to multiple articles together. It's more than possible to have a wiki with branching and cross-page atomic commits--it's even a good idea. But it wouldn't fit Wikipedia. And to get around the atomic commit problem (given the massively concurrent editing that happens on Wikipedia) you'd have to essentially have one Git repository per article, at which point you're probably better off with a different type of backend.

> Are approved revisions really a branch or just a tag somewhere in a linear version history?

A Git branch is just a different head, but no, those things don't have divergent histories. They're just moving tags. Anyway, semantics aside I meant that they're doing things now that maps effortlessly to a scm.

> Having n branches of a Wikipedia article would be frowned upon

Not if it was well supported. Saying "hey, I modified the article in my own private branch, check it out and merge it if you want" would beat a long argument on a talk page any day.

Branch support doesn't need to entail multiple publicly accessible (as in the stuff normal readers see) versions of articles.

> Also, Git-style branches only make sense as alternative branches of the entire repository

What? No, they'd also make sense even if there was one Git repository per page.

> you'd have to essentially have one Git repository per article, at > which point you're probably better off with a different type of > backend.

We won't know until we try, will we? But there are lots of advantages to using a well understood and simple data format with multiple implementations over a custom schema, but of course a custom format has its own advantages.

That's comparing apples to oranges. A proper comparison would be to create an individual Git repository for each article, not a giant repository containing all the data in Wikipedia.

That would work more like the current MediaWiki storage backend does, and would probably do delta compression of old revisions more efficiently than the current homebrew code.

While we're talking about Git-backed wikis, Gitit has been around for a little while now:

http://github.com/jgm/gitit

It supports Darcs and Mercurial repositories as well as Git.

Git Wiki (http://atonie.org/2008/02/git-wiki) is another one, popularized when al3x started using it.

Gitit has been around since 2008, but I guess that's a little while.

Actually, the more I read, the more this sounds like Gitit-in-Ruby-rather-than-Haskell.

Defaults to Markdown while support a bunch of other input formats? Check.

Drag and drop diffs? Check.

Git-backed wikis? Check.

Gitit still has more features (apparently there's no search for Github's wiki), but then, it's much much less popular.

Interesting, although, for what it's worth, bitbucket has had this for quite a while now.

So has Google Code.

Reply on news.ycombinator.com