Hacker News Reader

Comment by acabal | Hacker News Reader

Editor-in-chief here, happy to answer any questions!

My wife is an admin on the PG project. She puts in a lot of effort to keep the site secure. I think they welcome efforts like yours, and reuse is part of why they exist! They also research the copyrights, to make sure they are clear to share.

I use Project Gutenberg a lot personally, and in fact I'm using one of their books as a study for my upcoming startup:

https://github.com/carter-brothers/hand-propped#first-stage-...

One thing I really like about the PG site is that a few years ago they removed most of the JavaScript, which IMO makes it more usable. I haven't looked at Standard Ebooks yet, but I look forward to. Thanks!

One book I'd like to see on Standard Ebooks is "The Flying Girl", which my wife and I just read and loved. It's by L. Frank Baum (The Wizard of Oz), and is about a flying startup in 1911, which I previously posted on HN:

https://news.ycombinator.com/item?id=32142757

The Flying Girl would make a good first production, if you'd like to take it on! See https://standardebooks.org/contribute/producing-an-ebook-ste...

Make sure to thank your wife on my behalf for her hard work at PG!

What a great idea! There was also a sequel:

https://www.gutenberg.org/ebooks/53692

I'll look into doing these, I also know a librarian who might be interested.

I really can't believe Disney or Pixar, or some studio didn't make "The Flying Girl" into a movie. It really was bold and quite moving to see Baum (writing nom de plum as Edith Van Dyne) make the female character the hero, back in 1911.

first link is dead

Love this project! I’ve read several books from Project Gutenberg and owe a great deal to the project. I love seeing this extension of it and will be sure to avail myself of it and possibly contribute one day!

Not a question, so much as a suggestion from an interested netizen: it would be great to see changes flowed back up to Gutenberg. I know they have a process for submitting updates - I’ve made several to The Wealth of Nations myself.

Thanks for sharing the project, and I look forward to my next read!

It’s down to the producer, but 95% of the SE books I produce have changes upstreamed to PG. There’s no competition between us: we both serve different niches.

Yeah, the question is: why this, and not just put in with Project Gutenberg?

I think the differentiator is that SE puts a lot more care into the creation of the end product than PG.

I think that’s a little unfair on PG. Their goal is digital preservation, whilst SE’s is readability. We couldn’t do what we do without PG, and there’s (luckily) space for both projects to coexist in a mutually beneficial way. And the reader gets to pick a source that best matches what they’re looking for.

In general, I love the format, however I notice that (for example), Hugh Lofting's "The Voyages of Doctor Dolittle" has none of Lofting's illustrations - which seems a shame in a children's book - is this a deliberate policy choice?

Yes, we purposely don't include decorative illustrations in any ebook. (With some very rare exceptions.)

Thanks - as I suspected, one of those seems to be "The Return of Sherlock Holmes" where "The Dancing Men" story needs the illustration to make the story understandable.

What's the rationale for not including illustrations?

The ebooks we work on are often very old, and have been illustrated various times over the years. We don't want to have to pick and choose a single set of illustrations, nor do we want to forced to constantly justify exceptions for everyone's pet book, so we just have a blanket "no decorative illustrations" policy.

Note that this only refers to decorative illustrations, which is not the same as an illustration required to understand the text. As someone pointed out elsewhere, books like Sherlock Holmes or Agatha Christie, which can include narrative-critical illustrations like a map of a murder scene, or nonfiction which can often have relevant pictures but are not "illustrated" in the decorative sense, get to keep their images.

I would argue that, where the book is illustrated by the author, you should keep those illustrations.

(I'm against the no-illustrations policy in general, but illustration by the author seems like an especially clear case.)

yes, this seems to me to make sense, contextually. If you did Piranesi on architecture, it wouldn't make sense to exclude his drawings. or Ronald Searle's diary of his time in the PoW camps in Japan, or Edward Ardizzone's war diaries, or the sketches by "DD" watkins in his books.. or heaps of others.

arthur rackham's illustrations for childrens books, just like e.h. shepards for winnie-the-pooh, or Quentin Blake for his contributions to childrens books: its senseless to re-publish them without these illustrations (when the IP becomes available, of course)

I was imagining an illustration-free version of Dr. Seuss.

What about texts where there are more or less “canonical” illustrations? Like, for an edition of Bleak House by Charles Dickens, it’s just not the same without the sketches by “Phiz” that were present in the original publication. Same with John Tenniels illustrations of Alice in Wonderland.

Amazing project by the way. Can’t believe I just heard about it today, will dig into your catalogue with excitement!

I super respect your thoughtful principles. the focus has clearly paid off!

Is that just to cut down on file size?

even if they are also public domain?

Do you have any plans to support (and accept contributions to) non-English books? It’ll be amazing to see original Russian classic texts redone with the same level of care.

This is a common request and the answer is unfortunately no. Typography varies across languages and we are only experts in English typography. People have tried to start up various SE-like projects for other languages, but as far as I know none have taken off.

Fair enough. Would you consider officially endorsing any such projects (e.g. with links on your website), for mutual benefit?

What about Latin, it uses the same typography as English.

SE has a lot of English-specific tooling if you look at their cli tool.

What a great project! Would you have a need for a veteran web dev who also happened to be an English major? Took a look at the volunteer page, but nothing jumped out.

Absolutely! Creating epub ebooks is basically creating web pages. Epubs are just zipped up XHTML files, with exactly the same semantic structure and CSS styling you'd find on a well-made web page. That makes web devs with English majors our ideal type of volunteer!

Check out our step by step guide to creating an ebook[1] and then our Wanted Ebook list[2] for some good first-time productions. Then send a note to our mailing list and we'll help you through it.

Creating an ebook is a very satisfying endeavor - I always say it's like building your own lightsaber.

[1] https://standardebooks.org/contribute/producing-an-ebook-ste...

[2] https://standardebooks.org/contribute/wanted-ebooks

The list for accepting ebooks makes a lot of sense, but I noted that public domain books are excluded. What is the reason for that?

I’m neither associated with the project nor a lawyer, but I can speculate: afaik U.S. copyright law doesn’t explicitly allow someone to declare a text to be in the public domain.

From someone reading with a Kobo device:

* Have you considered putting the books onto the official store? It would make it much smoother to get books on the device. You could charge a small fee to pay for the effort, I’d pay the extra to support the project and avoid the hassle of doing an upload. I just saw above you now do bulk downloads, which will help also.

* Although your covers are beautiful, they only appear in a small corner of the screen for the Kobo devices I’ve used, even using the Kepub format, is that a known issue?

And also to thank you for the effort. This project plays a really important role and has been a source of pleasure for many of my friends and family.

We've been in touch with Kobo but they haven't expressed a lot of interest. We do have an integration with the Google Play store, so if you search for an ebook we have in our catalog, it should appear near the top.

I use a Kobo eink device myself and haven't noticed the cover art problem you're describing. Make sure you're on the latest firmware, and that you're transferring our kepub files using a USB cable and not Calibre. (Calibre may attempt to apply their own conversion on top of our own conversion, which can result in unexpected things happening.)

>Make sure you're on the latest firmware

If I may interject, overriding Kobo's firmware with KOReader (simple drag-n-drop of a file) gave a second life to my Aura 2. It's made it responsive and snappy while providing more features. It may not be necessary for newer ones (or for cover art on sleep), but I was on the fence to upgrade my 5 year old e-reader before trying it, feels like a new device now.

Are you aware that someone is selling your books on Kobo with the Standard Ebooks publisher name?

https://www.kobo.com/us/en/search?query=standard%20ebooks&ac...

Yes, that's going to happen. The books are in the US public domain and anyone in the US can do anything they want with them, including reselling them. Obviously we are not the ones selling these.

That's fair, but I'd think the issue was less about copyright and more...trademark infringement? Since they're selling as "Standard Ebooks", I think there's reasonable grounds for confusion that someone browsing the Kobo store might see one of the books being offered as coming from you and make a purchase as a kind of donation, thinking that it's going towards SE's upkeep rather than into some random's pocket.

This is a problem Project Gutenberg has had for years, on Amazon and other platforms. It's a game of whack a mole and ultimately not really worth pursuing. As soon as you shut one guy down, another one appears with the same great idea.

In any case almost nobody is buying these anyway, as there are so many other free ebook editions of just about all of these books already.

I'm not a lawyer, but I believe failure to defend a trademark is a great way to lose a trademark.

I'm surprised Kobo doesn't have a list of banned seller names for situations like this. It would take zero effort on their end, and they're clearly opening themselves to liability.

Off to create Amazon and Disney stores over there... /s

I am not a lawyer bit it seems that Standard Ebooks is not a trademark.

The stories may be public domain but your arrangement, and more importantly your trade name are protected. You may not have the desire to take action like a dmca takedown, but you're definitely within your rights to do so.

You can stop the third-party vendor from saying "I am Standard Ebooks", but you can't stop them from referring to the book as coming from Standard Ebooks, because that's true.

That is very sad to see.

Thanks for both answers, I use Calibre, so that’s the reason why I’m getting that.

I've taken to just opening the SE website in the Kobo experimental browser and downloading the books directly onto the device that way. The browser is pretty sluggish, but even then it only takes a few clicks if you know what book you're looking for.

There's been some talk about improving the Kobo browser experience, if you want to take a crack at it! https://github.com/standardebooks/web/issues/126

I'm getting old and my eye sight is going. Are there any read-a-loud options you can recommend? The Edge browser does a decent job on PDF's in a voice you can select, but it's kind of hacky for an entire book.

Have you tried using NVDA? It's an open source and free screen reader - https://www.nvaccess.org/

Thanks! This looks promising for the visually impaired.

I find it useful for reading out stuff on a page as well despite not having vision impairments, if you have a long form article you can just set it off and it will read it out to you.

if you have an Alexa it can read ebooks you've bought on Amazon.

the question is clearly asking how not to do this on Amazon.

the whole purpose of this thread is not to rely on Amazon

I enjoy the cover art selected for the ebooks. How do you select the cover art and what sources do you like to use?

Cover art must be in a fine art oil painting style, and it's up to each producer to find something thematically appropriate. Once they do we require proof of US public domain status. See https://standardebooks.org/manual/1.6.4/10-art-and-images#10...

Wikiart and Artvee are good places to start cover art research. Many museums now have explicit CC0 collections, too.

Finding good public domain cover art can be extremely time consuming - it's the part of the process most likely to make a new contributor give up. But when you find a great cover, it feels great!

Slight tangent, but have you considered using these new generative models like Dall E 2[1]? So far my experience has been that they generate great oil paintings. Maybe there could also be an automated pipeline that generates oil painting covers using book titles or first sentences.

[1] https://openai.com/dall-e-2/

Leaving aside the aesthetic considerations, the images produced from OpenAI software aren’t public domain, and so we can’t use them. See https://openai.com/api/policies/sharing-publication/ for a list of restrictions OpenAI places on the use of their generated images.

I'd love to see a section (or sorting option) with the most downloaded ebooks.

How many volunteers do you have? What motivates people to keep working?

I don't keep count but over the years I'd say the number is in the several hundreds, including one-time volunteers. We do have a core set of what we call "editors", who are volunteers in charge of managing individual productions, along with a solid amount of repeat contributors who work on ebooks as they feel like it.

I think people participate because it feels good to make something of quality, and then give it away. If you're passionate about literature, then you get to read a lot of great stuff while you're doing it, too.

To tack on, how could one join as a volunteer?

See here: https://standardebooks.org/contribute

Our number one need is ebook producers. If you're passingly familiar in a command line environment and have basic HTML/CSS knowledge, you can produce an ebook!

I’m curious what format the advanced epub is — epub v3?

Yes, the advanced epubs are just the zipped source repo, which is epub 3.2, including any "advanced" CSS selectors and so on. "Advanced" here just means that the epubs are written with a modern web rendering engine in mind, while most ereading platforms in the wild are still at an IE6 level of CSS/HTML rendering support.

The "compatible" epub is the same epub, but with various compatibility enhancements automatically added for different ereading platforms.

Currently only web browsers can render the advanced epubs at 100% fidelity. iBooks might be the only non-browser ereading platform that does a pretty good job with the advanced epubs, as I believe it uses modern Webkit as its renderer. For any other ereader, use the compatible epubs. (Or kepubs for Kobo.)

Would you have any Android reading app recommendation for advanced epubs? There are android ereaders, those should surely be able to handle any epub given the right app, right?

Lithium seems to be fairly good in terms of rendering our advanced epubs, but it's not perfect; note especially that it doesn't invert black-and-white SVGs correctly so unless you set it to a dark page theme, they'll be "invisible". There are also some other rendering quirks for things like very big tables used for play formatting.

Poor renderers are a perennial problem in the epub world and there seems to be little interest in improving the situation. Just use the compatible epubs - for reading purposes they're almost equivalent except in some rare edge cases. Or, you can use our 'read online in browser' option to read ebooks directly in your web browser, which will serve you the source of our advanced epubs. But of course reading in a browser is less than ideal.

Thoughts on introducing some kind of popularity/rank sort? I.e., number of on-site downloads, off-site citations or some book-equivalent-IMDB. Is it a conscious choice not to have one or just feature prioritization?

The site looks lovely and I think it's great to have classic books properly formatted for e-readers, I've snatched up the ones that were sitting on my reading list for a while, but I find it unfortunate that it's a bit rough to find new things. I guess that mimics the feeling of a library, where books are grouped by broad genre but only alphabetical (or random) within, but I feel like it would be useful if it had some kind of pointers for discovery.

I'd be open to that but there's limited time in the day! If a volunteer wants to discuss making it happen, send a note to our mailing list or open a GitHub issue and we can talk about it.

Thank you very much for demonstrating that a website without JavaScript is still one very good way to produce a website!

Have any of the ebooks included any serious math? I see that MathML is the expectation, but I wonder if the rules around the math aren't just ignored because of the rest of the culture around which books are selected and worked on.

Indeed, in academic writing I never see MathML used in an HMTL setting, it's always MathJax or KaTeX. For your purposes this is probably fine, but imagining if someone wanted to author a high quality math textbook following the same standard I would wager they'd run into a brick wall

Tractatus Logico-Philosophicus, Passages from the Life of a Philosopher, and A Tangled Tale are probably our most MathML-heavy books.

Books from the PD era are not very likely to feature math serious enough to require MathML. This is probably for the best... MathML support in ereaders is poor, with the exception of iBooks and Kobo. Raw MathML is retained in our "advanced" epubs, but it's converted to PNGs in our "compatible" epubs, for this reason.

https://standardebooks.org/ebooks/ludwig-wittgenstein/tracta...

https://standardebooks.org/ebooks/charles-babbage/passages-f...

https://standardebooks.org/ebooks/lewis-carroll/a-tangled-ta...

It’s certainly not “serious math,” but I’m rather proud of this revision I made to the MathML used for a throwaway equation in an obscure short story: https://github.com/standardebooks/fritz-leiber_short-fiction...

I saw you mentioned using a Kobo eink device. Is that what you would recommend? It seems the 11th gen Kindle Paperwhite and Kobo Libra 2 are pretty comparable. I'd like to support a non-Amazon product, but it also looks like I can get 20% off a new Kindle by trading in my very old Kindle Touch, making the Kobo effectively $50-60 more expensive...

What’s something you’d recommend to a friend as a beach read?

The Book of Wonder is a series of weird short stories, some of them gloomy and pessimistic but many with a thread of dark humor. "Chu-bu and Sheemish" is one of my all-time favorite short stories - it's creative and hilarious. [1]

I also liked A Voyage to Arcturus, which has become a modern cult classic. It's probably my favorite book I've read for SE, though it might be a little heavy for a "beach read." [2]

P. G. Wodehouse is always a good bet for lighter reading. [3]

The Martian books are also light swashbuckling sci-fi. [4]

The Murder of Roger Ackroyd is considered to be one of the best murder mysteries ever written. [5]

[1] https://standardebooks.org/ebooks/lord-dunsany/the-book-of-w...

[2] https://standardebooks.org/ebooks/david-lindsay/a-voyage-to-...

[3] https://standardebooks.org/ebooks/p-g-wodehouse

[4] https://standardebooks.org/collections/martian

[5] https://standardebooks.org/ebooks/agatha-christie/the-murder...

Thanks a lot for doing this! I just downloaded a PG Wodehouse book on the iPad and I am thoroughly impressed by the quality of the book! I read PG's books when I was young. I used to borrow them from a local library so I don't own any of them.

The first book I downloaded from standard ebooks was a PG Wodehouse book. I’d never read him and figured I’d give both him and the service a try. Neither disappointed.

Along with Wodehouse, I would also suggest E F Benson.

I’ll second Alex’s recommendations of P. G. Wodehouse (especially Jeeves Stories) and Agatha Christie.

A personal favorite of mine is Jules Verne’s Around the World in Eighty Days: https://standardebooks.org/ebooks/jules-verne/around-the-wor...

Would you consider creating a Goodreads collection/list of all the books in Standard Ebooks? Though I can appreciate if it's too much work to curate or manage.

Edit — oh, is this the list? https://www.goodreads.com/list/show/140305.Standard_Ebooks

We did have one volunteer adding items to Goodreads as we produced them, but the process was manual and very tedious, and the ROI was questionable at best. So IIRC he no longer does that. I'm not against the idea but someone has to volunteer to pick up the task.

Yeah it's a shame that Goodreads stopped giving out API keys, that could have helped with automating the list to some extent at least.

Do you plan to make other browse options? I'd like to browse by popularity or author. There's a bulk download by author, but not in the browse page.

Do you have a plan for ensuring that Standard Ebooks never gets overtaken by profit-seeking spam the way Manybooks was?

Love your project, thank you!

There are no guarantees in life, but as long as I'm at the helm I hope to avoid spam!

We do accept donations because as SE becomes more popular and attracts more contributors, managing the project is approaching the time required for a full-time job.

1. Apart from Project Gutenberg, where do your books come from?

2. When you proofread and fix typos, do you contribute the fixes back upstream?

The vast majority come from PG. When they don't it's another public domain transcription source, like Wikisource, Faded Page, or Project Gutenberg Australia. We usually don't create our own transcriptions.

Our producers can and do contribute back upstream! It's up to the individual producer.

Might be a stupid question but in France books over 50 years of age ended up "free" (to read not to exploit), does it work the same way on all countries? Does it mean we can access books that are over 50 years in your platform or even GP?

In Germany, works become gemeinfrei, a status similar but not exactly equivalent to the public domain, 70 years after the author's death.

Just a note, the line height (leading) when reading a book online seems too small.

Line height is set to the browser/ereader default - it's not something we change as it depends on the font. Font size is also the ereader default, though if viewing in a web browser we increase it slightly.

just curious, not planning...

Would you accept novel translations of non-English classics? The policy suggests yes, but I imagine such a work would fall under US copyright (which the policy forbids).

If by "novel" you mean "modern", then potentially, but they would have to be actually good translations, and released to the public domain via CC0. We only work on public domain books.

Hi @acabal. I'm interesting on a RSS address for your blog. I'm seeing you publish tech stuff. Thanks for all the work you've done and shared.

If you wanted to create an ebook from scratch, how would you do this? Would you write plain html. Would you write markdown, and convert it? What tools would you use?

How do you calculate the reading difficulty?

Using the Flesch-Kincaid scale: https://en.wikipedia.org/wiki/Flesch%E2%80%93Kincaid_readabi...

I wish there was a difficulty for non-native speakers. Flesch-Kincaid score does nothing for the majority of readers if it doesn't distinguish between Night and Day (Woolf) and Shakespeare's Coriolanus. From purely the language's perspectives, non-native speakers will struggle with Coriolanus much more due to outdated language.

Flesch-Kincaid makes little sense to me for books aimed at adults in general.

If I use Lithium Epub Reader Pro on Android, does that support "advanced epub" properly?

Great project thank you!

Have you considered adding A Million Random Digits with 100,000 Normal Deviates to the collection? It's quite the thriller -- every page is a surprise and you will never guess what happens next!

What's your favourite ereader on macOS?

I may be seeing this thorough a “sensitive” lens, but the language on the website feels like it’s bashing Project Gutenberg in order to show SE’s strengths.

> Ebook projects like Project Gutenberg transcribe ebooks and make them available for the widest number of reading devices. Standard Ebooks takes ebooks from sources like Project Gutenberg, formats and typesets them using a carefully designed and professional-grade style manual...

> Other free ebooks (which PG has already been highlighted as being in that category) don’t put much effort into professional-quality typography

It can be a very difficult task to compare without criticism. It’s clear that SE does put care and attention in to all the things mentioned with the goal of creating an excellent edition, I just think that PG has well-earned the respect they have for doing what they do well: getting so many books in to so many hands. As highlighted in the page and in your comments here: PG and SE fill different needs so there is room for both to stand tall.

That doesn't strike me as "bashing", it strikes me as contrasting different focuses.

PG is going for breadth—as many public domain books on as many devices as possible. Standard Ebooks are going for depth—a very quality of typesetting for each book.

Bashing? The goals of each project are different and compliment each other. I am almost positive the folks at PG encourage projects like Standard Ebooks.

Reply on news.ycombinator.com