Thousands of early English books released online to public by Bodleian Library | Hacker News Reader

Thousands of early English books released online to public by Bodleian Library

bodleian.ox.ac.uk

67 pointsdiodorus21 comments

Not exactly released. And not exactly to public. As the site says "Please note that it is very rare for us to set up free trial access for individuals. Individual pricing is not available and all trial requests will be considered on a case-by-case basis."

It seems to be available to the public http://www.bodleian.ox.ac.uk/eebotcp/, and on github at https://github.com/textcreationpartnership/.

That first site is remarkably poorly designed for actually finding the information, a common theme I find for websites created by librarians. There should just be a box at the top listing the download links!

Web developers working for libraries: Far too often I visit a site and and am confronted by acres of text explaining what the project is, who is involved, how to enter data in forms and all sorts of hand holding, BUT NOT THE ACTUAL DATA! Usually I find there's some link hidden in the least visible part of the page, like the lower right hand side, that actually lets me get started! Does Facebook have paragraphs of text with a welcome message from Mark, explanations of what Facebook is, how you use it, or does it just let you dive in and get started?

edit: And despite all that text it doesn't explain what phase I and II are.

I should say this is an amazing bit of work and it's really important that it's being released public domain, and a good sign of the direction libraries are taking. It's just a little frustrating that the final delivery step is so obfuscated.

I would like to point your attention to this: You don't have permission to access /cgi/t/text/text-idx on this server. Apache/1.3.39 Server at eebo.odl.ox.ac.uk Port 80

One of the problems searching in these books is that there is no standardized spelling in early modern literature.

There is a project called DREaM, at McGill to standardize for "distance reading" (macro analysis).[1] It uses a program called VARD (a text preprocessor trained to correct spelling).[2]

Strangely, this application is licensed with the creative commons. I think this means that it is closed source. Does anyone know of any open source alternatives?

It cannot handle such an immense amount of data,[3]

[1] http://earlymodernconversions.com/introducing-dream/

[2] http://ucrel.lancs.ac.uk/vard/about/

[3] http://www.matthewmilner.name/2014/11/18/VARD-and-EEBO-TCP/

There are lots of different CC licenses. None of them are closed-source, although there are some which don't allow modification (No-Derivs / ND). Others disallow copying for commercial use (NonCommercial / NC), and a couple allow almost anything (Attribution / BY, No Rights Reserved / CC0). https://creativecommons.org/choose/

> There are lots of different CC licenses. None of them are closed-source, although there are a few which don't allow modification.

Licenses that don't allow modifications are closed-source. At least, they are inconsistent with the Open Source Definition (specifically, with criteria #3: "The license must allow modifications and derived works, and must allow them to be distributed under the same terms as the license of the original software.")

http://opensource.org/osd-annotated

Oh right, I was just thinking of open publication.

I think that the distribution of VARD is allowed (if non-commerical), but editing the source code isn't. I think this because I cannot find the source for VARD.

That's why I think it's strange it is licensed CC.

They're using BY-NC-SA https://creativecommons.org/licenses/by-nc-sa/2.0/uk/ so it is strange that they limit access. I wonder if there are two competing groups, or if the legal department got paranoid.

15 years of work! Well done to those who funded and performed this extensive effort.

Files are hosted by github and box. Will Internet Archive be included?

For marketing this accomplishment, a few choice examples of long-inaccessible text may attract new readers.

Presumably IA can just get the Box and add the files to their own archive.

Not sure how IA works, but casual observation suggests they give upload accounts to libraries and the cataloging is done by the library, not by IA.

"The text files were created by manually keying the full text of each work, based on millions of digital facsimile page images"

!!! This is not silicon valley. I wonder how they ensure accuracy.

Link to the books http://ota.ox.ac.uk/tcp/

Well, maybe they used Amazon Mechanical Turk ;)

I know from a similar German project (http://www.deutschestextarchiv.de) and they have two independent non-German speakers transcribe the digital facsimiles to ensure that the transcriptions are as accurate as possible.

Hundreds of gradstudents.

I highly doubt that. One of the text [1] starts with the line:

> TO THE RIGHT VVORSHIPFVLL MAISTER RObert Clarke,

The mistakes look like typical OCR errors.

[1]: http://tei.it.ox.ac.uk/tcp/Texts-HTML/free/A01/A01716.html

In fact, those mistakes look more like accurate transcriptions of Early Modern manuscripts - with their looser spelling rules and often idiosyncratic use of letters.

It's kind of interesting that they look like the same errors as those generated by OCR.

The difficulty of deciphering the text makes this huge task even more impressive!

It's precisely those idiosyncrasies of early modern orthography which make it difficult to use an off-the-shelf OCR package, which is presumably why these are hand-transcribed instead.

Perhaps there is a specialist antiquarian OCR package which can deal with long s, interchangeable u and v, non-standardised spelling, etc, but I have yet to come across one.

Have you looked at The Early Modern OCR project? My understanding is that they're working on exactly that as well as simply better tools for reviewing & retraining on a large scale:

http://emop.tamu.edu/

No, I hadn't, and am grateful for the link - thank you!

Reply on news.ycombinator.com