Two approaches I see as plausible:
1. Use TeX as a "document assembly language" of sorts, a pure back-end rendering target not exposed to the user, and do your document markup/scripting in some other front-end language. Pandoc [1] is a start at building infrastructure for this. You can then build a new renderer too if you want, to remove the dependency on TeX, but at least in the meantime you have a pathway to producing quality output right now.
2. Build something on top of HTML+CSS (maybe also +JS). Advantages of this would be that it'd make it relatively easy to target the web with the same document source, and open-source web browsers have already sunk a ton of development time into rendering. CSS even includes a bunch of print-oriented features that in principle provide markup for most of what you might need here, though browsers for obvious reasons haven't tended to prioritize those parts. wkhtmltopdf [2] is a project aiming to build a to-print or to-PDF document workflow on top of Webkit. However imo the results are still not near TeX-replacement level. I believe the gold standard currently, if you want high-quality print out of HTML+CSS, is the proprietary PrinceXML [3].
I write a lot of TeX, and I have never found that those things are the fault of the standard software; rather, they are my fault.
The absence of a better alternative in the domain, even though attempts to serve the domain by other tools (including ones using more "modern languages") have come and gone over the years.
TeX isn't theoretically ideal, its just practically very good and the expected cost to benefit ratio of a ground-up replacement is very high.
TeX is really, really amazingly powerful. It can do almost anything a typesetter could want to do, fairly easily, and it can do just about everything, one way or another. And its output is heart-achingly beautiful. Sadly, the code necessary to achieve that output ranges from…heart-achingly beautiful to heart-breakingly ugly.
There are other projects out there, of course. I do think that TeX & LaTeX are close to a local maximum, if not al the way there.
XML, in comparison, is a booger joke.
A dried up booger on the floor.
Funny you say that since TeX's version scheme (in part) is that it approaches pi. And IIRC will be pi upon Knuth's death.
TeX is incredibly powerful, but it's also incredibly idiosyncratic. I also personally think Computer Modern is an ugly font.
No, it's really not. XML is a markup language, not a data-encoding language. JSON, S-expressions, ASN.1 &c. are all data encodings; TeX, LaTeX, HTML and XML are markup languages.
> TeX is incredibly powerful, but it's also incredibly idiosyncratic.
Agreed.
> I also personally think Computer Modern is an ugly font.
Eh, it's not great on-screen, but it looks pretty good on paper. But TeX & LaTeX have supported multiple fonts since the beginning.
If you replaced TeX, LaTeX itself would need replacing, as would every single LaTeX package and class that you ever want to use, ones like microtype and stuff, would have to be rewritten.
To be honest, the multi-pass deal isn't that bad, but the macro expansion system is crazy complicated. Every once in a while after working in LaTeX I'll get the feeling I understand it, but that feeling inevitably dissipates after ten minutes or so.
Making a "new" TeX probably wouldn't be that hard - but it's also something that wouldn't be that useful. I would very much like something that's both simpler and also keeps some of the lessons learned/implemented (word spacing/splitting, page layout, page breaks etc).
As for other "tools in the same space", I do like pandoc a lot. I want to like python's ReST (Re-Structured Text) - but that's a package I feel is in need of a rewrite/redesign. Many good ideas there - but figuring out how to take a simple document and produce simple, modern (preferably somewhat semantic) html for example -- or to produce a decent looking PDF without needing all of LaTeX/Texlive on hand isn't easy.
Rewriting ReST tools would be a lot of work, but I think if one didn't try for 100% backwards (output, plugin) compatibility it might be worthwhile.
The astute reader will notice that ReST/Pandoc deals with structured documents, and not really layout for paper/screen (both use TeX/LaTeX as an output target/pipeline). I don't know of anything that comes close to TeX/LaTeX for "rasterized" output.
On the other hand, I also don't know of any package/combination that'll make TeX/LaTeX produce anything but messy, 90s-style html -- that generally looks awful. Even if you were to try and force a modern set of CSS down over the resulting mess. If anyone knows of a modern hypertext package for TeX/LaTeX or some similar tool, I'd be happy to be proven wrong.
XSL:FO at one point seemed to have aspirations in that direction...