Project Gitenberg

> Perhaps Github would be willing to help set you up with your own instance of their platform which you could modify to better suit the purpose.

Alternatively, you could use, and customize, an open-source self-hostable alternative, such as gitorious https://gitorious.org/gitorious

> BTW, I recently learned the Gutenberg was not his name and is really a significant historical inaccuracy.

Do you have any source for this fact that Gutenberg wasn't using "Gutenberg" as his last name?

Anyone creating an OSS collaborative authoring platform could benefit from lessons learned in commercial authoring platforms:

http://blog.inkling.com/2014/06/problem-of-structured-author...

http://alistapart.com/column/wysiwtf

We need momentum around one integrated OSS toolchain, including illustration (Inkscape) and CMYK color for printing (Scribus). This would help editors and publishers who want to move away from rental pricing for authoring software.

As anyone who has tried to find good books in a sea of free books knows, there isn't a standardized way of collaboratively improving book Metadata. This needs to include e-production history, translators, print publication history and content objects _within_ books, viz. Doug Engelbart's purple numbers, http://www.dougengelbart.org/about/ohs.html

GITenberg is more of a reference to Project Gutenberg at this point than Johannes. But no, I didn't know that at the time :-(

Idea first, pun after. But thanks!

Funny thing was, about a year behind Seth, I thought up "Project Gitenhub" made a few repos, found GITenberg, and decided to direct my meager efforts in that direction. For sure the pun was a huge motivator!

PGDP does a series of passes on a book. They don't continually update it. If a book makes it through their process with a typo, or more common, made it through the transcription process 20 years ago, there aren't good systems in place to fix those issues.

Gitenberg and DP are quite different in their objectives. DP was created primarily to clean up OCR- text only. Modern ebooks have a whole 'nother dimension to presentation. fonts, graphics, reflowable layout, footnotes, links, indices, etc.

For example, I contributed enhancements to "Alice in Wonderland" https://github.com/GITenberg/Tenniel-Illustrations-for-Alice... (from a version in mobileread). DP produced one edition; There are in fact many public domain versions of Alice. How to keep them all straight? PG has no answer, but version control systems allow us to use fork and merge processes to start to deal with the way the real world works.

As a long time DP contributor, I'd say the models could not be more different. DP is elaborately and deeply organized around the model of a great many people doing a great many small units of work (stop by once a day and proofread just one page for very specific things). This minimizes the responsibility any one person needs to feel for a book. And like the citizen science projects at zooniverse, individual mistakes are corrected in a multi-pass process where multiple people (5 or 6) see every page.

The new project follows the Github programming model, which means if a person wants to contribute to a book she has to clone the book, make local changes, push, issue a pull request. This is far, far more complex than stopping by pgdp for a ten-minute proofing session. Very few of the drive-by proofers at DP could manage that technically, or would want to be that involved.

Most important it lacks the QC inherent in the DP model of having multiple later reviewers catching earlier errors. Who will do line-by-line vetting of the accuracy of pull requests? Besides the inevitable detail mistakes, there are potential problems similar to those faced by Wikipedia: who will even notice if some local zealot decides to insert editorial comments in, or to bowdlerize the language of, some classic?

It's true that DP's work ends when the book is posted to PG, and that PG has only a feeble update method (email to their errata contact). I could certainly see something like this project as an adjunct to PG, dedicated to continually refreshing the library, but with editorial control over which books go into it.

+1, in code the basic unit is usually a line, but in prose it is the sentence. I've tried using git with LaTeX and it always ended up with weird situations where you had to put each sentence on a different line to make it work effectively.

That exists, and it's made by a YC alum, Nate Kontny. It's called Draft. https://draftin.com

Check out http://prose.io/ it might be suitable - its aimed at the CMS end of things though

Git is actually great for text (code is just text after all), it's just a matter of a better display for the diffs. Might want to have a look at a project I'm working on, Penflip [1], which does exactly this. It uses Git to track changes in markdown files, and has a web interface like GitHub but geared towards writers instead of developers. The in-browser editing interface is a fork of http://prose.io/, which allows for git commits (and other actions) from the browser.

[1] https://www.penflip.com/

I discussed this with one of the Git maintainers; it seems that the main work needed is a modification of the way text is chunked (into lines), and this chunking is already suitable factored.

text editing on github is pretty easy now for non git users - it does all the branching and pull reqs for you.

> But maybe it should pick a clone (gitlab ?), self host and fork/extend that tool to ease the use so that non-developer could use the site without git knowledge

We're on the same wavelength here. I forked GitLab to make a 'github for writers'. Still backed by git repos, but a simplified web UI for less technical users. If you're interested in working together, lets chat.

Crowd-sourcing translations for video games is a fairly common practice these days. I haven't used any of them myself, but people have developed specialized tools to make it easy for community members to contribute translations. Perhaps it's worth looking into what they're using?

GitLab B.V. CEO here, ask indicated by the author of Penflip in the other comment, GitLab is indeed a good start to build something like this and we welcome initiatives likes this.

In retrospect, I didn't need to make 80k+ commits with my own account.

I imagine it working similar to the original Project Gutenberg site: http://www.gutenberg.org/

For example, known books by established publishers, but with a self-publishing arm http://self.gutenberg.org/

The main thing that GITenberg could provide authors is a toolset and workflow for using git to store books, and some kind of toolchain to turn them into epub or print-ready pdf.

That toolchain is something I effectively have to build for GITenberg anyway.

Gah! This website was thrown up quickly. I wasn't intending to post to HN until after I had fixed up the website. Someone beat me too it :-S

Pull requests welcome: https://github.com/GITenberg/gitenberg.github.com

Please please please don't use "black". Pure black looks awful compared to a dark grey: #2e2e2e is my preference, but #222 is close approximation of black that looks very nice as well.

Like others, I am doubtful that this is the best way to go about it.

But to answer your question, the main area where I've found Project Gutenberg's epubs could be improved is in their navigation outline (the toc.ncx file). For example, they often use top-level headings for each line from the title page, then put the entire book under the last line. Whereas other books are closer to what you'd expect, albeit at inconsistent levels of detail. For my project, I abandoned their TOC's altogether and created a simpler format.

The images are also at a bare-minimum of resolution. In some cases, higher-quality versions are available in the public domain (such as on Wikimedia Commons). Most of the books are also scanned on archive.org, and so can be referenced there in facsimile. These tend to be higher-resolution scans (although those are all monochrome that I've seen).

For corrections in the works proper, I have occasionally submitted corrections by email but never received a response.

Otherwise, they are perfect, and I thank them for their outstanding work.

EDIT: There are also rare cases (I think Seneca was the one I came across) where the id's are not unique across the book, even if they are within the HTML files. I couldn't find anywhere in the EPUB specification that would require this, yet for practical purposes I think they should be made unique across the book, since the division into HTML files is arbitrary.

Further to that, there are some PG books that have a unique (serial) ID on every paragraph. Again, this is not required, but it's extremely helpful when it's there (for anchor referencing). It would make the whole library more usable if this were applied consistently, and the serial id's are apparently mechanically applied.

There are a number of transcription errors in many books. There is an example PR https://github.com/GITenberg/Chess-Strategy_5614/pull/1

I've noticed several problems recurring in Gutenberg ebooks. Mistakes in the words per se are rare, but:

* substitutes for characters missing from ASCII (e.g. L for £, no proper dashes)

* incorrectly delimited chapter heads (e.g.:

    *CHAPTER 11: THE GREAT*
    BOONDOGGLE George Boondoggle sat on his lawn...

instead of

    *CHAPTER 11: THE GREAT BOONDOGGLE*
     George Boondoggle sat on his lawn...)

* footnotes appearing in the middle of a page (based on the pagination of the original print edition, perhaps)

* missing italics, underlines, etc.

* ASCII-fied equations and diagrams

These do not detract from PG's original goal of being an archive of plain text, and suffice to provide scholars of the 22nd century a good view of what was written in the 19th, but they do detract from the experience of somebody who just wants to read Anna Karenina for fun. (Especially if they are a typography nerd like me)

From another perspective, one repo should allow you to gain more traction as all stars/forks/pull requests/commits will be aggregated on it, and thus produce higher visibility on GitHub (and probably anything that scans github stats).

Additionally, using a single repo would allow me to fork and specify my own styles that I want applied to any work I "compile", and these might be hyper-specific.

I'm actually willing to help consolidate these repos if you're willing to go in this direction. I'd also like to hear reasoning for multiple repos if there's something I'm missing.

Because parsing the original metadata from Project Gutenberg is time consuming to write. I wasn't going to submit it to HN until I had an index/search api, but someone beat me to it.

> Every time a programmer has an issue with Git, whoever helps them has to sit down and explain the underlying system for 20 minutes and draw a bunch of sticks and bubbles.

This isn't true at all for a lot of people. I know a lot of people that just read the docs and are able to solve the issues. Others will Google the problem and find the solution on stack overflow. Everyone learns differently...

> Anytime someone shoe-horns it into a product they talk about how Git is so amazing and solves all these problems, but what they are really talking about is just a version control system, not Git specifically.

Git is amazing and does solve a lot of problems, but there are problems that aren't solved by Git. Even Linus himself says this here: (https://www.youtube.com/watch?v=4XpnKHJAok8).

Using the github API, rather than git, for creating epub books and pdfs is a great. Using git to control changes as the do is perfect as well.

> Non-programmers will never put up with this.

Ermm don't assume that everyone gives up right away. With the GUI interfaces we have today, Git is really simple once you learn it.

(As an aside, I sometimes feel the same way about node.js, where I've seen "node.js is awesome" listed among a project's "features." Nothing against it, I just don't get the obsession.)

I appreciate this comment with respect to Git right now. I've recently spent a lot of "hammock time" trying to come to grips with my views about this profession generally and what I believe is best going forward. One thing I feel strongly about is that while we are still maturing as a field, the pain points are unacceptable. There is still so much work to offload to the machine, requiring fundamental rethinking at many levels. So although I agree in principle with the initiative to help people "learn to code" (so that we can bring system design closer to the domain experts), I also believe that in the current state of things, it's a wasteful effort, since it requires conveyance of ideas that should be deprecated.

But even short of programming, version control alone would be useful in so many other fields. There's no reason why it shouldn't be a mainstream concept even for personal use (e.g., you're working on a thesis). Just an hour ago, during my annual flirtation with Git (I'm a Mercurial user), I wrote in my notes:

> the barrier to entry for new programmers is important. This would appear to weigh in favor of Mercurial — and yet, realistically, is a “layman,” i.e. someone who knows nothing about software development and has never used a CLI, really going to distinguish between these two systems, or will the very concepts of a VCS not prove to be the biggest hurdle?

I have used Git, and I think that for linear history the differences are not remarkable. But the attitude you refer to is crucial: do we want to hide complexity or expose it?

Incidentally, I have several Project Gutenberg epubs under version control for a personal project, and like the OP I attest that their work is first-rate. There's no comparison to any other digitizer in the public domain (that I know of).

I can't think of a VCS with a better online tool than git and github. With editing books, it is entirely possible to use only the github editor, which effectively abstracts the git command line interface.

Writing version control software is hard. There are just so many potential use cases, not to mention the differing perspectives on how users interact with each use case as well as how they're applied to specific projects a user is working on, increased complexity is inevitable. On balance, I think Git manages to strike a good balance with most things even with its unique eccentricities.

Git's popularity isn't because it's the best tool out there for all scenarios. It's popular because it's a distributed system that helped communities grow around code managed with it while removing barriers to entry. In my opinion, that more than anything will be Git's lasting legacy.

Git, in spite of the horrifically complex interface, is in essence a really dumb version control system (I mean that in a non-insulting way). This means it's fairly neutral about what kinds of data you can throw into version control. And more importantly, it almost never complains about what you give it.

I think that's why people are now starting to think about applying version control to domains outside of code and choosing Git to do it. For example, I had an idea a few years ago to make a CMS on top of Subversion as the data store (never got around to building it though). Now there are lots of projects like that built on top of Git: CMS, Wikis, you name it. Generally anything that can work off flat files is very easily converted to use Git as a back-end, giving you advanced version control features more or less for free.

From a practical perspective, the difference is not just that Git is a trendy new silver bullet, it's the "dumbness" that makes it actually easier to do that kind of work than it would be on older version control systems like SVN. Interestingly, for the most part, most of these projects do not really benefit from the distributed nature of Git (although for things like wikis and CMSs it can offer yet another feature: content migration between instances). It's more about the ease of use for getting data into a repository and under version control without it exploding when something unexpected happens, like a file getting renamed.

You might not get the best front-end experience for actually doing stuff with that version history (as other comments have noted wrt diff tools, etc., which tend to be geared towards code rather than other types of content) but that's the fault of those tools rather than Git (which is dumb enough not to care about content types), so it's just a question of incrementally building up a better toolset for your particular content domain. That's much easier to do and more approachable than building the whole infrastructure from scratch.

As for Github, it happens to have a nice interface, toolset, documentation and mindshare. Developers are familiar and comfortable with it, so there's no need to research and learn "yet another tool". And because it's cloud based, you can get up and running very quickly without worrying about hosting, etc. That's just more icing on the cake really.

> Every time a programmer has an issue with Git, whoever helps them has to sit down and explain the underlying system for 20 minutes and draw a bunch of sticks and bubbles. Non-programmers will never put up with this.

But from my experience, they have to do this exactly once per (non-stupid) programmer. The moment you grok underlying structure (basically all graphs and pointers), the apparent complexity disappears and most of the things in git become obvious. I see no problems with explaining this to non-programmers as well, you just have to spend a little more time, because they probably aren't used to think in terms of graphs.