Ebooks, you’re doing it wrong
blog.thomasrhiel.com
blog.thomasrhiel.com
It's not always the publisher's fault--I've developed a few ebooks for different platforms, and ebook readers today are where Mozilla and IE were around the IE5 browser wars. Developing a book that looks good across all platforms is difficult, if not impossible in some cases.
It wouldn't suck so much if we weren't paying almost the same price as a print book. Why would I pay ~$13 for a brand-new, often terrible-quality ebook (and a DRM'd one at that), when I could buy two print books for the same price, and get Amazon to ship them free to my door?
I'm not trying to say you're wrong or that I even disagree, I'm just trying to point out that people should have different expectations with ebooks and physical books (like I mentioned in my other post which was downvoted). ebooks are awesome in some ways physical books are not (and they're lousy in some ways that they really oughtn't be).
The point I most strongly agree on is where you say this is in its infancy (like the browsers wars). The first ebook I read was on a Windows PocketPC in about 2000 and it sucked. The whole technology has gotten dramatically better and I expect it will continue to, but it's not more than about 10 years old, really, and only really started gathering steam within about the past 5 years. And I'm not saying "therefore we should all have low expectations" but I, personally, am willing to look at all the things that have been done well and focus more on those than focusing on typography or whether editing is immediately correct.
Why are books even OCRed to create ebooks? Do they not have the original digital copy of them to work from?
So they're not willing to provide the original source files, or what?
> edits late in the process are made to the typeset InDesign files, after the structure has been lost.
Wow, they do this so haphazardly that they don't even bother to go back to the original Word file and add the same edits?
Is InDesign used for all books? I don't see why you would need it for something without any figures/graphs/tables. You could just use Word, and things would work out fine.
How about effectively dealing with ligatures?
I know that people like to harp on and on about how great LaTeX or InDesign are, and when it comes to complex layouts (particularly LaTeX if there are lots of referenced figures and InDesign for text wrapping, etc.), I agree. If you were writing a scientific article, I'd tell you to use LaTeX. If you were creating a magazine, I'd tell you to use InDesign.
But for your typical, text-only, fiction novel, the kind you're most likely to read on a Kindle? I don't think it really makes a difference.
(And even if they weren't, I would appreciate such examples.)
People make a big deal about how using Word will lock you into a certain format when they're storing their data on floppies or, even worse, rasterizing it and keeping just the PS. Seems like the latter is a lot worse than the former.
I guess they could generate PDFs from Word files with Acrobat and shoehorn them into the process, but any publisher that is actually satisfied with doing that is bonkers.
As for why OCR is used, just look at the range of books that are now available electronically. There are a lot of books from the 1980s and earlier that are now available as EPUBs. Do you really think that it would have been easier to use the source files off floppies? (Keep in mind that the floppies are, for the most part, long gone.)
How are reprints of such books achieved?
Typography is more art than science.
What is the incentive for better typography? Most books are like mini-monopolies (over the short term). If the typography of a book sucks and it sucks on all platforms, what is a consumer to do if they want to read the book? Fallback to print? Publishers probably don't mind that. It's true that all things being equal, typography could be a deciding factor for the consumer, but my gut says that in most cases the content is weighted much higher.
I think the platform could benefit from typography. A consumer may choose an iPad over a Kindle if the iPad had a better reputation for typography. But since there is _art_ involved it might not be economical for the platform to pursue it. Which is a better selling point: Our platform has 100 really nice books or our platform has 100,000 books?
As far as your final hypothetical goes I honestly do not know what answer you were expecting, but maybe that's the point?
I believe this is true in general. A few savvy people catch on that a product is better and it becomes a meme that others rely upon for their decisions.
I do not think that there are enogh savvy-typography-valuing-people to induce some market driving meme. If this was the case Word would have died many years ago...
You would think the same people who design, manufacture, and market a product can have their shit together long enough to take some great beauty shots for e-tailers. But nope. The vast majority of vendors are so poorly organized, so bureaucratically frozen, that getting them to do anything that isn't already part of the process is like pulling teeth.
So, "please send us digital copies of this manuscript" is pretty much out of the question. The easiest way is to take a copy of the book, OCR it yourself, and then get the license to distribute it.
The customers would then be able to customize the font sizes/stylings, etc., and use those settings for all the ebooks they read.
Two printed paperback books, with their horrible sand-paper pages, pixelated fonts, and crease-forming covers... surely you can live with an out-of-place hyphen if you can live with a paperback.
I dont know - and neither does reddit/selfpublish . How does an author go from writing to generating books without encountering specialized (and expensive) typesetters?
The first piece of advice that self-published Kindle authors will give you is to have a beautiful cover - how do you design one and integrate it into your ebook generation workflow to get one nice little package ? Can I say that the publishers are fleecing us, when probably they are having to pay extra to make sure their ebooks look good on Kindle, Nook, Sony, etc. ?
The toolchain doesnt exist - and it is time for disruption.
I buy ebooks because they're easier to see... The lousy formatting makes them less enjoyable to read in some cases, but seeing is a prerequisite to reading. I need the inverted screen and the ability to enlarge the text and view the book on a back-lit display.
I've also noticed that my daughter, age 9, has an easier time reading ebooks with the text size bumped up than she does printed books. In her case, it's not about eyesight, but the fact that larger text and fewer words on the page makes it easier for her to focus - she retains what she's read better. There may be a slight tendency toward dyslexia or something there. It's not sever enough to get a diagnosis, but there is some sort of little "glitch" there that both we and her teachers have noticed it.
Instant delivery is another nice feature. If I decide at 10pm that I want to start a new novel, I can have it in my possession by 10:02.
So ebooks offer many advantages in the realm of accessibility, and there are certainly many more books available in ebook format than there are large print editions, but the poor typography and high price are certainly annoying.
Awful technical decisions such as: text justification and margins are fixed and inconsistent, spacing between words is awful, footnotes break the "navigate to last page" feature, pictures show up in the wrong place, tables look awful, charts and diagrams are full of jpeg compression artifacts) and so on.
It's really infuriating. Nowadays I check all 1-star and 2-star reviews for mention of "ebook" or "kindle". That usually keeps me from making a bad purchase. With the combination of lousy typesetting and comparatively high prices amazon really shows contempt for their customers.
There does seem to be quite a bit of inconsistency on the part of the publishers, though. They complain that ebooks are destroying their business models, and then restrict Amazon and other retailers to charge comparable prices for ebooks, but they don't treat ebooks comparably themselves. Ignoring the fact that releasing an ebook doesn't require physical printing and the multitude of material and labor costs associated with that, they don't seem to apply the same editorial (and typographic) effort that they do with printed books.
I'm hoping that they will try (and catch on) a ebook+hardcover release model. After all, people buy hardcover either (a) because they need the product now and can't wait for paperback (or there isn't one), or (b) because they also want to show it off in their personal library. The ebook+paperback could offer that experience. Hardcover $30, Hardcover+ebook $33. That's $3 for free. Oh, hmm, looks like O'Reilly is already is doing this bit :-)
A month after each new release, you make paperback and ebook available simultaneously, offering the ebook at minus publishing and shipping costs. ($5 paperback, $3 ebook?)
"Almost half the kindle ebooks I buy are formatted so
poorly I can't (sic) read them."
I really wish that the typesetting was better but I am not sure if this is something that we will see any time soon. Can you imagine running latex on the kindle cpu everytime you flipped a page? On the other hand the ebook files would be huge if you prerendering all of the pages. You would have to prerender a separate page starting at every "location" and in each font size. That is an enormous number of combinations.
Probably not for a whole book, but for a single chapter? Almost certainly. And if you eschew the features of XeTeX or LuaTeX and just stick with pdfTeX, things could be very quick: For the example I posted elsewhere in this thread of typesetting a novel in 30s with XeLaTeX, pdfTeX does it in about 6s. A single chapter (32k of text): 0.8s, and I/O bound at that. Even if a tablet's 800Mhz Cortex A8 does that 5 times slower than my Core 2, it's fast enough to be done speculatively. It's certainly no slower than loading a web page over the internet.
(For comparison, when run on the slowest CPU I have handy, a 1.5Ghz Athlon XP, the single chapter takes 1.5s wall time and 1.1s CPU time.)
What slows TeX down is that many packages use extremely elaborate abstractions, which all need to be evaluated using string expansion. These days you shouldn't be surprised if some package is using floating point emulation with TeX macros (https://github.com/latex3/svn-mirror/blob/master/l3kernel/l3...) behind your back. Handy yes, but certainly not fast.
Illustrations missing or put together in a block in the middle of the book.
Tables converted into justified text,equations missing, source code reflowed into an obsfurcated paragraph
1) Shitty rendering engines used in (some) ebook readers. If the ebook original has an em dash and it's being converted and displayed as hyphens, that's just bad rendering. Same thing if it has two hyphens together in a prose text context surrounded by spaces, and it's not being shown as an em dash, bad.
2) ebooks that start out bad. In this case, it's garbage in, garbage out. For this, the blame goes to the publishers, as he correctly noted.
Has anyone done side by side comparisons of the same books, from the same publishers (I'm not talking about Gutenberg books that may have been converted using differing technologies, but major publishers), on different ebook readers from different companies?
...but does that mean that we do the same in our day to day jobs? The code that you have to fire out to meet next Friday's deadline? The program that will make it into the next product that needs to hit the market now? I'd bet almost all of us know the feeling of, "ah, if only I had a little more time, I know exactly how to code this the way it should be". But this is business so (a) the time you need just isn't there and (b) what the heck, you don't care as much about the accounting app your company sells anyway.
So, yes, I'm sure that ebook publishers aren't idiots. And they know that ebooks can be done differently from printed books. But they are businessmen, too, and so the question is always: will doing these changes give them more profit? That is, will the additional costs of e.g. moving the glossy-paper pictures from a central position directly to the parts of the books that refer to them result in more sales? Or sales at a higher price?
I figure if the answer were yes, the publishers would do it. But as long as they get away with sloppy editing and one-to-one copies of paper prints, why would they?
The additional costs are not material costs, but that people have to sit down and make these edits. And that's not just one guy spending half a day to do that.
this will also change when more and more people are opting ebook version to the printed copy. as of now the printed copy sells more compared to ebook format AFAIK and the ebook format is catching up fast for the many convenience stated in the posts here. We can expect a change soon as many people are getting ebooks and raising their concerns.
this could also change when some really nice product(device or software) to read ebooks comes to the market, I dream of a ebook reader where I can annotate, mark and make notes using a stylus like device. while many of these features are available in the ebook readers today they don't give the same feel as using a printed copy, for example (making handwritten notes)
But rationally speaking, if someone who bought the ebook tells the publisher that they would have liked it better if it had feature X, the incentive for the publisher to implement that feature is still pretty low - after all, the guy complaining bought the ebook anyway.
It may have a long time effect though, because the same customer might not be willing to buy the next ebook from the same publisher unless the perceived annoyances go away.
And competition from other publishers might help too.
It's chicken and egg. Until there's a large enough market, they won't care. Low quality negatively influences market growth.
Of course it's unfortunate that consumers are now left with sub-par product, but I think it was necessary to get the ecosystem going. Beautiful typography and ebook-appropriate formatting will get there, just give it a little time.
Current tablets should be able to re-render an individual chapter in a second or two, and it would not be any trouble for ebook distributors like B&N and Amazon to pre-render portrait and landscape versions at several different type sizes.
Fixing the abysmal typography of ebooks really wouldn't take much work, and would probably be the best thing ebook publishers could do to woo serious book-lovers who aren't also geeks. Why aren't they even trying?
The only thing I can think of is that normal people, even book-lovers, don't care. We have been trained by reading webpages so much of the time that we have low standards. But the sad thing is, ebooks generally have worse typography than your average blog post read in a typical web browser. I'm looking at you, iBooks. Jesus.
There is a beta 'microtype' package for XeTeX but it doesn't have all the features that LuaTeX has (font expansion?).
Lualatex/pdflatex are the future.
I take it that your 800k novel does not have an index and/or a lot of intra-text references. 30 seconds would be a great time with indexes and references. Why don't you use fontspec too?
I don't believe that you need to render a new PDF for every possible location in the book. It's ok for a hyperlink to point to something in the middle of a page.
Why are we talking about PDFs now? If you seek to a location in the kindle it takes you to a page with that location at the top. That location might be in the middle if you read from cover to cover.
The reason we're talking about PDFs is because that's the quickest and easiest way to get good typography on screen. And being able to scroll to put an arbitrary line at the top of the page is not a killer feature for a book replacement, and certainly doesn't make up for a book that is hard to read due to poor line spacing, justification, and kerning.
As far as PDFs go what ebook reader uses pdfs as the native ebook format?
Out of curiosity if you set the paper size to ebook in memoir how does the compilation time change?
\documentclass[twoside,12pt,openany]{memoir}
\usepackage{fontspec}
\usepackage{hyperref}
\hypersetup{bookmarksnumbered}
\setmainfont{Baskerville}
\usepackage{graphicx}
\usepackage{microtype}
\raggedbottom
$time lualatex *.tex
real 0m16.498s
user 0m14.291s
sys 0m0.395s
real 0m18.642s
user 0m15.243s
sys 0m0.449s
real 0m5.946s
user 0m3.601s
sys 0m0.179s
real 0m11.652s
user 0m5.449s
sys 0m0.250s $ time tex texbook.tex
Output written on texbook.dvi (494 pages, 2029384 bytes).
Transcript written on texbook.log.
real 0m0.615s
user 0m0.584s
sys 0m0.024s
What is wrong with ebooks is that there are still books that I _cannot_ buy. I want to give the publishers/writers my money and they simply will not accept it. For me its e or nothing, I don't have the space to store paper books, and to be honest I just like ebooks better.
Books and newspapers don't have the same typographic rules. The web have other rules. Ebooks probably have other yet-to-be-invented rules.
But I agree with the claim: most of the books I have read on my Kindle have bad typography. It should be fixed on two sides: The reader's software can be improved[1], and ebooks have to be proofread and adapted to this medium.
[1] Oh I wish Amazon could iterate on the Kindle (I mean the real one, with e-ink), improve its reader software, its browser, etc., instead of running after iPad's success.
Many of these books are eventually corrected and Amazon will email letting you know this so you can get an updated copy, but this has generally been months, or years[1], after I've put up with the problems and finished reading the book.
And it would be nice if pictures were handled better. I think ideally they should all be put at the end of the eBook but have links in the text that take you to the relevant picture, so you can quickly look at it and hit "back" to go back to the text where you left of.
[1]: Just this morning, I got an email from Amazon telling me a book I purchased in May 2009 has been updated with corrections.
(It purports to take the work out of Gutenberg->Latex->PDF; I've not used it, simply had it buried in the back of my head somewhere)
With that in mind, I convert the Gutenberg etexts to a textual format my eBook will show nicely (joining sentences to paragraphs etc), see http://www.michielovertoom.com/python/gutenberg-ebook-scrapi... , source on https://github.com/motoom/gutenberg-ebook-scraping
I write popular fiction right into Tex and use pdflatex to typeset my books. But that is also not a solution. As error proof as it is, it is rather cumbersome and not very user friendly.
The problem of (most) of today'e ebooks is not that they are typeset in Word, but that they are converted from Word.
Contrary to common sense, authors' write (produce text) in Word (WYSIWYG editor -- not text) and then they upload their .doc files into epub creators to produce epubs (text). Yes, we go from text to text via something that isn't text. Bound to be a few glitches.
Word has hidden commands that an average user cannot anticipate unless they are surgically careful with their formatting. Most of today's epublishers have written an entire book length instructions on how to do this error free. It's exhausting.
In my opinion, writing, as a process, must be revolutionized for this to be fixed.
There are ways I think it works poorly for prose, though, too. The most obvious example to me is the George RR Martin Fire & Ice books. When I'm reading those, I very frequently want to flip from a section of the book to the maps of the world described in the book. I read a physical copy of the first two books but went to Kindle (both the 3rd generation Kindle and Android app) for the third and it's tedious to set a book mark, navigate to marks and notes, find the map, and then navigate back to the bookmark. Additionally, the map images in the ebook are of such low quality that they are nearly unreadable. Lastly, there's an appendix of the families represented in these books at the back (who's who). When reading the first two books, I often would consult that, too, but once I'd done that with the Kindle book, it made it really painful to switch back and forth between reading on my phone vs. my kindle, because the "sync" function is "sync to furthest page read" - which is at the end of the book, not my current place in the book.
All of the above gripes with that book would be plain to see for anyone who bothered trying to read the book on a kindle - and it should be obvious to the kindle team that "sync to furthest page read" breaks when a book has an appendix / index or something that readers are likely to consult.
I did get into the habit of bookmarking my current page to reference the maps and appendix. Generally my biggest problem with "sync to furthest page read" was reading while underground (subway). Usually I just fire up the iPhone briefly so it will sync forward if I want to read on a different device.
There is also the "Book Extras" section which, for Ice & Fire, includes content from Shelfari with some appendix info and does not break the furthest page read. It would be nice if the official appendix and maps were included in a similar format.
I haven't noticed the "Book Extras" you mention and will have to investigate - thanks for the tip!
I guess once I grok ePub a bit better, I can editing the texts to change the code samples' formatting.
Also annoying: It will landscape PDF's, but not ePubs.
/grump
Recalibrate your expectations a little - buying an ebook is a lot like buying a physical book, but it's not the same thing. Maybe I should start blogging about how unhappy I am with the portability of physical books or how I was in an airport and wanted to rip the first chapter out of a book in the store to read on the plane but the store owner TOTALLY wouldn't let me!
Which is NOT to say that I disagree that it matters.
I just expect it's going to get better, because the platform / technology is relatively immature. And I think it matters a lot less today than people who blog about things like this (or those who downvote dissenting comments about things like this rather than trying to be persuasive) might think.
I mean, there are many very bad physical book publishers out there that don't (or minimally) edit the copy sent to them before sending it to the presses. That translates into the digital realm too, but there is a much lower barrier for entry for bad publishers and it's hard to differentiate between bad and good publishers in these early days.
If no publisher out there is doing a good job, it might be something to invest time into and maybe make some money.
Also, when you are talking about ebooks, shouldn't things like font selection be left for the reader (the being, not the machine) to decide?
Our ebook is for the open source/FOSS project PyGame. http://www.pygamezine.com/
PyGameZine is in Beta, and we're launching today.
We made the book in html, and converted that into PDFs. The process was quite hard over all. Haven't started on the process of turning it into an EPUB yet.
However, we felt that authors would find html easier to write, and designers would find it easier to design in.
Maybe something like restructured text would have been ok to do too. Since it is a python magazine, and many python programmers would know restructured text.
If we make it to another issue, we might think about using a separate toolchain.
> With a little effort, the makers of The Publisher’s digital incarnation could have one-upped the printed book by positioning these photographs to accompany relevant bits of Brinkley’s text, illustrating, e.g., a description of Henry Luce’s childhood with a photo of Luce as a toddler.
I don't know anything about typesetting, but this specific example seems difficult to automate with software. A human would need to decide where best to place these photos.
Again, a guess, but it seems that this book started life in a pre-electronic book production process. To get it right would require more manual effort for the e-book.
Just like websites are evolving to be adaptive/responsive, I think book production will follow. We're just not there yet. Websites are still being launched today that are close to impossible to read on a mobile phone.
Remember, TeX started out with the goal of being able to automate the typesetting of a very large, very complicated multi-volume book (The Art of Computer Programming). Novels are almost too easy, and adding a few pictures to a biography is mostly trivial.
I think going forward, the publishers will plan for typesetting the print and electronic versions together so the delineation of costs that I mention won't exists. It will all be rolled into the production cost of the book.
My comment is aimed at publishers with piles of books that were set before the existence of ebooks (or with production processes that are slower to change, books in transition). Should the backlog of books be converted to e-books as quickly as possible using a less than ideal processes or should the books be manually re-set with humans thereby delaying their availability?
I think the publishers are trying to optimize based on highest return between typography and availability.
BTW, I would love to see better typography, I don't disagree with that point. I'm just speculating on why it's not happening.
I can hear it. Meanwhile someone is rubbing his hands (or should I say wallet) and tells people that they are now available on ebooks.