The price of a messy codebase: No LaTeX on the iPad
vallettaventures.tumblr.com
vallettaventures.tumblr.com
I disagree that the current situation is LaTeX developers' fault. TeX and LaTeX are complex pieces of software, that are developed over time. They were extended as the capabilities of their platforms increased, to take advantage of those capabilities. This requires full access to the OS utilities, and naturally LaTeX environment does that. If anything, this is Apple's fault. For whatever reason, they cannot allow applications to use the existing capabilities of the underlying OS. This goes against the Unix mindset; of course there will be unpleasant consequences, but one cannot hold Unix mindset responsible for these.
Do note that the same is true on Google's Android OS. You can run as many threads as you want, but no calling other executables to do work in the background ...
LaTex dates back to the beginning of the 80s. Given that it has the ability to create large documents, I doubt that the required amount of main memory scales linearly with the size of the document.
Also, when you use subprocesses, besides checking their return value, you have some sort of interprocess communication which should detect when one process failed to finish his job.
It looks more like the smartphone runtimes and their sandboxes just don't support it at all, because modern GUI applications rarely build upon multiple executables in the background.
In any case, I absolutely agree that resource constraints aren't keeping LaTeX off of our mobile devices.
Of course to make the software useful in the real world, a lot of things have to be added to it - make sure to not include any of it in the 'core' otherwise the 'bug free' claim will go out the window really quickly.
So yeah, in that sense I guess it's technically (which I use as a euphemism for 'autistically') correct to call TeX bug free (the bounty program for bugs found in it is quite famous, and part of fore-mentioned marketing).
Really?
He's made a career for himself out of being an extremely creative computer scientist and writing outstandingly scholarly books about programming. Far from marketing himself as producing bug-free code, he has published articles entirely devoted to the history of bugs found in his software.
(As for the bits that turn TeX into LaTeX, no one is making a career out of claiming that they are bug free -- for the simple reason that no one thinks they are.)
The point those packages make is that some problems can have a solution that lasts years or decades instead of mere months.
That being said, I also think you have a low bar for definitions of "bugs". Terrible performance is very much a "bug" in my book. Even the things that aren't "bugs" in the strictest sense can still make for bad software. The executable described here is 4GB, that's 1/4 thte total space on a low end iPad! Also, if tomorrow someone does discover a bug, it sounds like this would be a mess to fix. The jumble of source simply won't be attracting many new developers (whether it be to fix existing bugs, add new features, or port to new platforms).
Really, borrowing MikTeX's install-on-first-use for packages and making documentation optional should result in a much more reasonable size. If you want to get fancy, teach LaTeX how to decompress packages on demand.
For the record, I do not own devices based on iOS or Android and I don't care for your little Apple-Google feud with which you seem to be so concerned with. So please, carry on with those market statistics about Android, I am dying to know more.
To get around it, you'd have to make it In App Purchases, because these also come from a the App Store.
From that quote ("A 4GB TeX distribution dependant on over 100 binaries is not acceptable on the iPad."), it is clear that he is musing about a scenario where he ships all 4GB. As stated many times (and also in that same quote), on the iPhone you are forced to ship something as one executable. Thus, the theoretical distribution he is referring to would in fact be a single executable of about 4GB. In fact I think that's the whole point of those two sentences. So it appears that I was indeed quite careful.
On the other hand, yes the executable must be "one". On the other hand you can download stuff (either directly or via DLC). The issue is that while Apple has relaxed their restrictions on bundling interpreters in applications (you can do that), they have not relaxed their restrictions on downloading interpretable code: it's not allowed. So downloading even styles would be verboten, since styles are TeX, and TeX is executable code.
AFAIK Apple doesn't restrict you from bundling files in your application. That simply wouldn't make any sense.
The point here is that he is talking about an application consisting of a single runnable binary of size X with tons of bundled .sty-files that brings the whole application up to 4GB.
Previously, this would not be allowed, since .sty-files are interpreted (which in turn would make it impossible to implement LaTeX on the iPhone), but that really wasn't the point here. The point is simply that this wouldn't be a single 4GB executable, but a an executable of perhaps 100MB bundled with a whole bunch of .sty (and other) files to a total size of 4 GB.
Edit: Fixed typos. Sorry. HN doesn't work very well on my phone.
I'm interested to see how this could be justified, instead of "hit'n'run" style.
Noting here that other than the source problems, performance was a considerable issue.
Yeah. The only benefit is a codebase that reflects current needs and can grow in new directions and follow current practices, that programmers of today are familiar with. And also be faster, and more flexible. But what are those compared to compatibility right?
(Of course people interested in pure compatibility could just continue to use the old version, until the new is compatible enough, or they don't need compatibility anymore).
"""If one tries to rewrite TeX, the problem gets much much worse, since TeX is, for all intents and purposes, bug-free."""
And your wrong assumption is a) that a problem has to be bug free to be useful or b) that a new effort can't produce a relatively bug free codebase?
How are either of those requirements wrong? I certainly expect my programs to be bug free. LaTeX is a tool that's older than many GNU utilities. It's been reviewed and tested numerous times over by the harshest test suite of all: the real world.
Yes, in theory, you could rewrite TeX to be cleaner, while maintaining backwards compatibility and lack of bugs and regressions. In practice, this is very very hard. It's so hard that no one, no individual or company has managed to pull it off.
But hey, who knows? You could be the first. The LaTeX codebase is open-source. If you think you can pull off what the most talented software engineers in the world haven't been able to do, go right ahead.
Good luck with that. What I said, though, was that "a problem has to be bug free to be useful" is a wrong assumption. We all use programs that have bugs, and still found them immensely useful.
"""Yes, in theory, you could rewrite TeX to be cleaner, while maintaining backwards compatibility and lack of bugs and regressions. In practice, this is very very hard. It's so hard that no one, no individual or company has managed to pull it off."""
What are you talking about? Tons of people have written TeX versions of their own, even in Lua, and most of those are widely used.
"""But hey, who knows? You could be the first."""
Hardly. There is LuaTex, XeTeX (even more widely used) and others. Are you just talking out of your ass? (Not to mention several other modern typesetting engines that are not TeX based. You make it sound like TeX is an impossible achievement).
Now, what I said I'd like to see was not a rewriting of TeX itself, but a modern version of a TeX like engine, with or without compatibility. Programs like XeTeX and Omega for example provide Unicode support and TTF/OT support to TeX, but I'd like to see something more modular, and with a more streamlined markup than TeX+LaTeX. While we're at it, even scriptable.
There are also some improvements found since in typesetting, like those employed in the InDesign layout engine concerning edge text and hyphenation.
The purpose and benefit of backward compatibility for TeX is that a 30 year old manuscript written in TeX can be fed to the latest version and produce identical output to what was produced 30 years ago. (Of course, you'd also have to archive the macro packages and font files you were using, but those were never machine dependent).
The notion that every time we have a new language/platform, we must REWRITE THE WORLD!! is absurd and a huge waste of resources. We should stop making excuses for it.
Yeah, you probably missed the whole point of the comment, TFA, and several other stuff besides.
It's not Apple's platform that's the problem, is TeX/LaTeX that is lacking important features. In EVERY platform.
People have been hacking them on to it, but having a better codebase to start off, without the assumptions TeX/LaTeX make would really make it so much easier.
"""The notion that every time we have a new language/platform, we must REWRITE THE WORLD!! is absurd and a huge waste of resources."""
Yeah, good thing that nobody suggested anything like that in the thread.
Btw, the notion that you can add your worthless reply without first understanding what the parent poster says is absurd. You should stop making excuses for it.
Using the native capabilities of the OS in the sense you mean - allowing TeX to use its ancient, hideous bitmap font system, goes against:
- Unix philisophy - do one thing well, link against or pipe to other apps/libraries that do their thing well
- Simplicity. The user should not be forced to make an unnecessary choice where one option is almost always superor to the other.
Knuth's concept of 'bug free' is one of an academic computer scientist. TeX has many, many issues: unnecessary dependencies, a horrible build process, and many UX issues that stopped TeX from reaching TeX's original goal of 'allowing anyone to create typeset documents'. This is fine: Knuth is a mathematically focused computer scientist from a period where everyone who had access to a computer was a programmer. It was valid in this era.
All kidding aside, I use (La)TeX almost every day and it's bulletproof. It's not going anywhere, so you might as well learn to love it, worts and all.
TeX's original goal of 'allowing anyone to create typeset documents'
Actually, according to this interview[1], speaking of the very first version of TeX, "I should mention that I had no idea that I was designing a typesetting system that anybody else in the world would ever use, besides me and my secretary. My idea was that I would teach Phyllis, who was my secretary for almost 20 years… So Phyllis and I were going to be the users of this TeX system. … I tried to design it just good enough to do my books and no other books."Even with the released version, I don't think TeX was ever intended to be the last word on typesetting; it was simply his best attempt at a decent markup. He then froze the features so that previously typeset books could be corrected and reprinted using the same source. He left it up to future programs to innovate.
But I don't think Joel is against a slow, one line at a time, working on the old code base rewrite.
It is not clear what kind of re-write the OP is advocating. But I would be surprised if he is pro the start from scratch kind.
But it might be just my biased perception.
I find it likely that you'd be better off performing an algorithm extraction and doing a true from-scratch rewrite than trying to incrementally translate TeX and LaTeX.
For the reader's convenience and reference: http://www.joelonsoftware.com/articles/fog0000000069.html
* LaTeX is working on the iPad, see for example: http://meeting.contextgarden.net/2010/talks/2010-09-14-arthu...
* LuaTeX (http://luatex.org) is written in C, not WEB.
* You only need one binary to work with TeX. Either PDFTeX or LuaTeX. All other binaries are just glue code (for example to generate missing Metafont fonts, but who wants them these days anyway?
A lot of TeX uses Computer Modern to typeset the body (a metafont font) and I don't think I have ever see a TeX document that doesn't use Computer Modern or Euler (another metafont font) for equation typesetting.
I also wonder whether it's possible to achieve the excellent equation typesetting capabilities of TeX without metafont.
I can think of no reason to prefer CM to LM for typesetting purposes.
And that's before getting into anything truly gross.
If I needed LaTeX on a daily, here's what I would do:
1. Install DropBox on my mobile device and a designated "build" box--maybe just your home computer or an EC2 micro.
2. Set up Jenkins on a box that's got your favorite LaTeX environment and create a job that takes your LaTeX source files, builds and outputs your desired formats and pops them into an output folder you created on DropBox.
3. Set a build trigger that runs a build every time you touch a specific "ready to build" file.
4. Edit and write LaTeX source files on your mobile device using an app that syncs with DropBox like PlainText.
5. Increment something in your "build trigger" file.
6. Et voila, check the DropBox app on your mobile device for output and log files. :)
The crux of the problems with porting it seem to be that everything has to be in 1 executable, that there can be no scripts, and it must be in an approved language. That's merely a problem from Apple.
A correct title would be "The price of restriction on apps: No LaTeX on iPad"
The author also thinks it's be easy to just switch licence to GPL. Unless LaTex & TeX required copyright assignment, you cannot switch to GPL without getting every contributor to agree. The author is also under the believe that you cannot have commerical software that's GPL. It's DRM software you cannot have with GPL, and that's why Apple doesn't allow it.
No. It's a problem with LaTeX & messy codebase.
Take Apple out of the picture, for a minute: the messy codebase remains a problem with TeX/LaTeX.
It doesn't lead itself to easy extension, it doesn't lead itself to easy incorporation of features such as full Unicode support, Open Type font special capabilities, and much much more. There have been several attempts to fix this, namely LaTeX3e, Omega et al, but none has been that successful. We're stuck with a codebase that works, but is messy, too complicated (ever tried to set up a LaTex/Tex distro from scratch), and obsolete in many parts.
Once you are talking about a multiple GB system, of course it's messy. Except it is actually a very poor choosing to describe the situation. E.g. mixing multiples programming langages in a big system has nothing messy, it would even be a HUGE warning not to encounter multiple programming langages in such a huge system.
Only there's no need for a typesetting engine + layout templates + graphing, math, bibliography etc add-ons, to ever be a "multiple GB system".
It just carries too much legacy garbage.
I know a lot of the above are supported (I use LaTeX myself, and follow the developments quite closely) in newer TeX implementations, but not in clean way and not with first class support, to the legacy limitation.
I was asking for a clean implementation, no something like XeTex --that while supports several modern features it inherits tons of design problems from the original TeX.
A long-running process (daemon) is therefore a much more efficient way to run (La)TeX.
[1] www.tug.org/TUGboat/tb27-0/fine.pdf
""" On my current 800 MHz PC, the command
$ tex story \\end
takes about 0.137 seconds, while $ tex \\end
takes 0.133 seconds. The first command typesets a small page of material; the second does nothing but start TEX and then exit. Thus, typesetting the small page takes about 0.004 seconds.
"""Which is not to say it's not a good idea here.
Admittedly, getting everything linked into a single binary might be fairly hard.
Sorry, but I fail to understand, why any serious TeX-Head would say that. The platform is crippled and your problem shows it. You are ranting in the wrong direction.
-jsl
But I am not sure if "fault" is an appropriate term here anyway, we're simply observing two different approaches to software bundling that happen to be incompatible.
How can anyone really be suprised that one of the oldest and mightiest gods of one religion can not be coerced into the shrine of a completely different religion?
Personally, I'm in the camp of "rely on as little as you can". This includes not shelling out to bash unless you really really really can't achieve the same thing with a bit of more code. Apart from making the app less portable, more bug prone, less maintainable etc, it's just ugly.
I use my tablet to write — quite a lot — way slower than on my regular keyboard, but with time to think. Everything is in text files, synced to my desktop where the typesetting is really performed.
Writing publications or books is a long process where basically the final rendering is taken care at the end because the quality of the wording is more important.
So, just a text editor with hard work on the words, sentences, ideas are where 99% of the time is spent.
Maybe the author can provide an extremely pleasant way to write on the iPad and on the Mac with syncing of the work to have the feeling that the work is never lost, that one can always update a bit of the manuscript and just run "make" on the Mac some times to times. Everything in a smooth workflow.
You can pair the iPad with a bluetooth keyboard and write away at full speed.
And with something like LaTeX you won't even have formatting to slow you down.
I understand that tablets are a wonderfully portable device but you can't expect to get 100% desktop functionality. Sometimes you have to find the limit of your software/hardware.
I don't own an iPad, but is there git or the like? If so syncing and keeping track of changes would be simple.
The point is that there does seem to be a software limit, but there shouldn't need to be a particular limit there. There's certainly not a hardware limit, since an iPad can run circles around almost all of the computers of the era when LaTeX was written. Unless you believe that the TeX system represents an almost magical, once-in-a-lifetime, impossible to replicate achievement in software engineering, there's shouldn't be a software limit either, but the code base is so complicated that there does seem to be one.
That, or Apple changes their policies.
Many people seem to think that tablets functioning as "laptop replacements" means duplicating all laptop functionality on tablet hardware. I take "replacement" to it's full meaning, in that a tablet allows you to develop on a server while making no compromise on portability.
If you wants to carry around the full LaTeX installation, you need a laptop. If you want a laptop replacement, you can't expect to not replace your way of working.
What on earth is in there?
http://www.amazon.com/Computers-Typesetting-Volumes-Boxed-Se...
[looks sadly at own unread copy sigh]
The base system was 280+ mb along with 190+ mb of documentation. I found it mildly amusing that I had to download ~500 mb of stuff just to update my cv.
Things like inserting graphics with text flowing around them (picins), or document classes for resumes,or textpos (for absolute text positioning), or......
Basically you have nearly two thousand such packages in TexLive......
So, the TexLive distribution is the "fire-and-forget"-package in the LaTeX world, thats why it is so big. I use it for around 10 years now and I never had to install an additional package.
[1] It even prohibited that for a while, see Wikipedia.
Isn't latex from the 80's? I think my first disk that had more than 200Mb is from the early 90's? How did people use it back then? Also it is still horribly slow even on a core2 with 4gb ram and superfetch.
Should we do all this work - mandated by business-driven 'app store guidelines' - simply to allow known good software to continue to work?
All in all, here is my opinion: who ever upvote this should feel ashame of yourself for wasting everyone's real-estate on this stupid rant.
The author states that even with the restriction lifted, it wouldn't make much sense, as with all the extension that it requires to meet the author's need it would take 4GB.
The situation would be pretty much the same in Android, despite it being rooted in Linux. The article is not so much about Apple as it is about LaTeX.
If you would like to make a point, at least read the article.
Rendering to DVI or PDF is something else entirely. While other comments here indicate its doable, I have to ask why? I render my documents on another machine (I actually have a script that monitors dropbox and periodically re-pdflatex's if the timestamp changes -- redirecting error messages to a file is trivial if you need to troubleshoot remotely). If you're attempting to do LaTeX with a WYSIAYG interface or (worse) doing the edit-a-line-rerender-repeat cycle, I'd have to say that you're doing it wrong. The assertion that "doing LaTeX" (or, frankly, any platform that supports simple text editing) requires the native presence of the entire LaTex codebase is just misinformed.
Finally, the author of the post apparently started his project without knowing that the original TeX distribution was written using literate programming, what kpathsea was, or that there were other implementations of LaTeX. Perhaps they weren't quite prepared/qualified for the undertaking? This smells like a classic case of "this isn't how I would do it/use the tool, so it must be broken".
Why do you want to hurt yourself typing TeX on a virtual keyboard? If you consider to add a real keyboard to your ipad then you should better get serious and use a Macbook, Linux or Windows.
Yeah, that's what TeX does.
It does not solve the problem of writing TeX on an iPad offline, obviously.
Putting it on a server makes the architecture more robust against messy code (and changes in programming languages and environments), because pieces can be changed independently.
The problem is roughly equivalent to producing a non-upgradable perl. It's not very useful on its own, but it's completely impractical to include a copy of the entire CPAN with it. And Apple's policies prevent you from downloading additional packages. It's just not worth the effort.
Computer science and engineering departments should use these situations to train their students.
There is some truth in your microrant here. I would restate it like this: Instead of getting excited with how expressive is a new sexy multi-injection functional language when you write the nth Fibonacci sequence, I would think it more realist and constructive to get excited with how a new language or a new extension of an existing language allow to handle module and package dependencies in a clean, safe and explicit way.
It resonates also with my work today: I spend a few hours removing those evil "import *" from our codebase... (Python's "we are grownups" mantra is nice, but I think a piece of the quote is missing: "we are grownups, thus we run pylint and comply to all the rules, except a few carefully chosen ones")
Seriously, they discovered that after starting the project???
Maybe you just need a smaller laptop ..
Also, terminals are great, but writing anything serious without X is a pain.
> Maybe you just need a smaller laptop ..
No, I don't want to have a small laptop as my main computer. I had an Eee before the iPad, but the difference in the writing experience is not too different in the end and the other advantages of the iPad are greater. In any case, you can hardly call me a person who unreasonably hangs on to Apple stuff: I'm writing this under Linux which I installed on the MBP my university gave me. I hardly ever boot into OS X here.
> Also, terminals are great, but writing anything serious without X is a pain.
Why? Emacs doesn't really need X, and I don't need to re-typeset my work every five minutes.
Guess this depends on how you work. I often find myself starting many instances of the editor[1] which is a pain if you have to create a new ssh connection for each (or use ctrl-z etc).
"the writing experience is not too different [..] the other advantages of the iPad are greater."
Guess the real difference between us is that you see something great in the iPad, so you are willing to live with its obvious shortcomings[2] compared to a laptop. I have yet to discover why it's so amazing, and only see a small, but pretty much useless gadget. It could be used for reading I guess..
[1] yes, it has tabs.. still
[2] low resolution, can't run what I want to run, can't type on it without a keyboard, and at that point you could just as easily carry a small laptop (not an Eee ffs, a real laptop).
Well, ok, I use screen to detach irssi etc.. but not as a "window" manager
Oh, absolutely. If I were a actual road warrior, the iPad wouldn't work at all for me. However, between my laptops at home and at work I don't really need a third one, and since I already have the iPad I'd rather make as much I can with it for short trips where I am not going have too much time and/or wifi to pull sustained work sessions.
> It could be used for reading I guess..
That too. For an academic, GoodReader is a blessing. I have finally stopped printing out loads of articles and/or finding that I printed out this same article a couple of years ago. Not to mention Kindle etc. Again, if I were spending significant amounts of time away from either home or the office, I'd consider investing in a proper portable laptop; but I'm not, and for my use case the iPad is ideal.
> can't run what I want to run
Yep, I agree that it's frustrating.
> (not an Eee ffs, a real laptop).
No everybody can afford another proper laptop that's as easy to carry around as an iPad, y'know.
Anyway, YMMV, of course. I was just pointing out that solutions are available.
If you want to use a 'walled garden' device like an iPad, you can't really complain when you run into artificial limitations.
A two page LaTeX test document on my old Dell Optiplex (Q6700 cpu) with Ubuntu 10.4 takes 0.033s, two orders of magnitudes less than he experiences.
The 539 page reference manual for the project I'm working on takes 3 seconds to format.
Then good luck! Please post again once you've finished the rewrite.
Does it have a good selection of packages, or just the bare-bones?
[1] Codify: http://itunes.apple.com/us/app/codify/id439571171?mt=8
If you want to do anything that looks remotely like work you don't want to use iDevice for it.
Oh wait.
The other applications pages, numbers, keynote are severely limited, and really slow and difficult to use. They are OK for viewing documents created elsewhere, but not so much for creating new one.
I'm assuming it's been ruled out for other reasons though.
EDIT: this would also make it trivial to use it on android/blackberry, without any extra effort. apparently, also, http://www.scribtex.com/
I wouldn't sasume it has been ruled out - people (myself included) sometimes get mentally stuck on one solution and fail to spot another obvious avenue.
Maybe the Lout system will pick enough traction one day.
Lout's equation formatting is especially primitive as I recall, and that is a huge amount of work to get TeX-like quality for. I am unaware of any system that typesets mathematics as nicely as TeX does.
It would be good to see UTF-8 natively supported, and for all the common packages to be absorbed into the core libraries. Right now, you can sometimes find 2 or 3 different packages that do the same thing, often with varying benefits. Take the best package, add the missing features from the other packages, and bundle it all into one distribution. For example, the absolutely horrible table support, where merging cells is a nightmare.
Viewing the output would be trivial: convert the output to PDF. For the ssh client case, push the PDF to a cloud file service like Dropbox. In the native client case, have the daemon push back the PDF for native client to render.
LP means treating a program as a piece of literature. A literate program is free from the structure of conventional source code layout; this is different to simply documenting source code.
I don't claim LP is a hot topic, but I think to say it's obsolete is to misunderstand the concept.
What I would like to run on the iPad, on the other hand, is a TeX-syntax-aware, document-validating editor that knows how to consume a rendering web service that runs on a Real Computer.
Xterm (iSSH) to linux box? ~ http://news.ycombinator.com/item?id=280730
Why have no one suggested to split LaTeX into two parts like the Emacs server, or something?
With a client and a server to run on your desktop/laptop/etc.
Edit: Clarity.
It's definitely not perfect--I really wish there were something better--but for writing technical academic papers, it seems like far and away the best.
LaTeX is about as far from dead as the desktop computer.
Hilariously, they screwed up the implementation, which does not match the specification in their tech report: http://tex.stackexchange.com/questions/20487/why-does-ms-wor...
And if you're including mathematical formulae, which scientific academia will do, HTML is rubbish for formulae.