Hacker News Reader

Comment by logicrook | Hacker News Reader

> I expected that everybody would publish latex files (raw source, not PDFs) in computationally accessible formats, with raw data in easily parseable, semantic forms.

Do you realize that many researchers would be ashamed if others saw their latex code?

A friend worked for a journal, he kept a folder of the worst that he had ever seen, and could rant for hours one some gems. Including some papers that were beautifully typeset, about elegant combinatorics/type theory, and were coded on the latex equivalent of a spaceship cobbled with recycled plastic bottles, scraps and duct tape.

Do you imagine going through thousand lines macro tex file that grew over the years, when Latex is one of the worst mess ever, with conflicting packages, absurd syntactic 'features', and legacy crippling flaws that make debugging an horrible task?

Please, continue to compile your .tex.

View on HN

> Do you realize that many researchers would be ashamed if others saw their latex code?

I really don't get this argument; I see it come up regarding open data for reproducibility too. TeX/LaTeX source is just an extra piece of metadata which might help someone out in a few years time. That's nothing to feel ashamed of, and it's certainly less shameful than not providing the code. If the code allows flaws in the research to be exposed, that's either something to be proud of, or (if it was intentional) it's not the code that's shameful.

My own LaTeX is mostly an inconsistent mixture of copypasta from stackexchange. That doesn't mean I'm ashamed of it, and it's all in public Git repos at http://chriswarbo.net/git if any human or bot actually cares to look.

Speaking of which, the commit histories of those Git repositories aren't very good either. Lots of commit messages like "Fixes", or "Oops, typo in last commit", with multiple changes bundled into single commits, atomic changes spread across multiple commits, etc. But, in the same way as my LaTeX source, it's better than having no commit history, which would be the case if I set myself higher standards.

Of course, I can follow higher standards, but that's when I'm paid to and/or when collaboration is important. For personal stuff, I'd rather invest the time I save interacting with Git on writing more tests, or refactoring, etc.

It would be much easier to get better if you could view other's latex code.

And if others could contribute improvements on yours.

I will never understand why latex devotees think everyone else should be really into typesetting as a hobby. It's like sneering at photographers who don't want to mix darkroom chemicals.

You have obviously never used troff then.

That's not a good comparison, because mixing your own chemicals for photo development adds more time, effort, and defect rate to the process.

Becoming more proficient with TeX saves time and makes for more re-usable typesetting structures, which allow you to accurately do what you want faster and faster.

It's similar to using emacs or vim. You could say, "Why should programming devotees expect others to adopt those editors?"

It's about productivity. I'm a productivity devotee and when I hear people use short-term thinking to discount how valuable the up-front costs for learning TeX (or emacs/vim, etc) are, it motivates me to help them see why it's not true.

Of course, I don't want to be dogmatic either. If something works for you and your personal utility function discounts the value of things like TeX or emacs, that's fine.

> It's about productivity.

No it is about aesthetics, which are mostly irrelevant for 99% of all research papers.

It's not just aesthetics, having proper typesetting makes papers more accessible, easier to read and parse, the standard formatting makes understanding the layout of papers easier, there is a huge library of packages that cleanly abstract the task of rendering obscure symbols and complex mathematical formulas, and it automates the job of tracking figure, equation and reference citations to name just a few of the features. Fine, TeX is an ugly language that only a programmer could love, but in over twenty years nobody has yet been able to replace its expressive power and flexibility -- doc generators just compile to LaTeX and more user friendly GUIs like Lyx and TeXMaker have been written to ease the process of writing for end users, but the underlying format has yet to be replaced in over twenty years. Like Markdown, TeX rides the line between being able to be written by hand for greater flexibility and automatically generated for newbies.

Unless you think that reading red text on a blue background in payprus would be an equally enjoyable reading experience it more than simple aesthetics.

... did I just read a rant about how enjoyability and simplicity are NOT aesthetics?

Are you related to the author of "The Joy of Tex"?

You sound like aesthetics is a bad thing you need to disavow for some reason.

> Fine, TeX is an ugly language that only a programmer could love, but in over twenty years nobody has yet been able to replace its expressive power and flexibility

Which doesn't make it less ugly. If we can't cure common cold yet, that is no reason to praise it and making it sound as if it's a great thing. It is a problem and so is the need to use ugly language to produce aesthetically pleasing papers. The fact that we don't have solution for this problem yet does not turn it into non-problem. It just says nobody yet was good enough to solve it.

In my field, equations and diagrams are relevant for 99% of research papers. I would poke my eyes out with a grapefruit spoon if I had to spend the rest of my life trying to do this in Word.

It's about productivity for the 1% or 5% of stars who understand it and eat it and breathe it.

For the 90%+, it's an arbitrary checkbox requirement to be checked on the path towards something else.

I don't eat and breathe LaTeX (although I try to use it for any typesetting needs: after a few years in academia it's much faster to do it this way than to open "Word") but the productivity enhancement on writing while forgetting of how it looks and just having to think about how you want it to look and letting the system handle it for you is invaluable.

> It's similar to using emacs or vim

Actually, that's a great comparison. I don't use either of those (yeah, I said it) because of their arcane interfaces. I don't doubt for a second that wizards with those programs, and LaTeX, can do great things, but IDEs exist for the same reason word processors do - to make it easier to focus on creating.

Yes, but if you pay the up-front cost of learning the interface (which, far from being arcane, is actually designed the way it is because it's ergonomic), then you can focus even more on creating than the person using the IDE. While they are stuck debugging some autocomplete error in Eclipse, you've already finished your code project, wrote a letter to your mom, dabbled in an interactive R sessions to teach yourself R, and played Tetris, all without touching the mouse.

IDEs exist for the same reason 7/11 exists. Convenience. You pay a higher price (lost productivity) to experience certain comforts (clicking a button instead of issuing a keystroke command).

My experience has universally been that IDEs only get in the way of focusing on creating. They might be necessary when you are literally just beginning with software and you need a lot of scaffolding to help you -- and I don't think anyone would begrudge university students using IDEs.

But not using a power editor once you are an experienced programmer is hard to understand. I mean, it's so important that it was even included as a whole chapter in The Pragmatic Programmer.

Paying the up-front costs of proficient editor usage is like paying the up-front costs of wearing braces so you can have straight teeth or wearing correcting shoes so you can walk properly. Of course it's more convenient in an absolute sense to simply not wear braces or corrective shoes. But that's not the point. The point is that the value you get from the outcome (e.g. straight teeth, proper walking, or dramatically increased productivity when developing) is far greater than the cost.

In other words, I think you're using a hyperbolic discount function. You are placing such a high value on absolutely immediate "focus on creating" that you incorrectly discount how much more focus on creating you would be able to do with the power of a real editor framework.

It's a trade off. Getting proficient with a new subject requires time and mental space, one that could be otherwise used for doing other things, usually things much more interesting to you unless you happen to be excited by doing TeX stuff. That's why one would usually opt for "just do whatever is needed to get over with it".

> It's similar to using emacs or vim. You could say, "Why should programming devotees expect others to adopt those editors?"

Exactly. There were times when I wrote emacs lisp and vim macros, and had time to dive into arcane mechanics of it. I don't have that luxury anymore. I need the editor to just work and do what I need to be done, without me investing significant time into it. Maybe if I spend couple of days on learning stuff about vim I could do in one minute when now I do in ten, but I'm not sure it pays off anymore, and frankly I have more interesting things to learn. I do use vim (not emacs anymore) but for me I came to the conclusion that added value of diving into the arcana with the goal of being more efficient later vs. brute-forcing through the immediate tasks is just not worth it. Later you'd need a different thing than you thought you'd need anyway.

And, note, I am a programmer. I can imagine how less interested would be a person who does not spend their days digging into the code anyway.

Becoming more proficient with TeX saves time and makes for more re-usable typesetting structures, which allow you to accurately do what you want faster and faster.

Not necessarily, there's an opportunity cost to all the time you spend become proficient with TeX. If there are other products that offer sufficient control over typesetting with less time and effort, and you don't anticipate needing the advanced features that are unavailable in competing products, then becoming a TeX expert is a complete waste of time. I'm happy to concede that it's probably the best tool and certainly the best value for money, but if someone's needs are quite limited then an adequate-but-easy-to-use tool may be preferable to the superior-but-user-unfriendly one.

At the very least it would make sense to provide the source that outputs the document. That could be something like OpenOffice or Latex. There are probably some WYSIWYG solution for Latex. If you are referencing a table from another paper and you want to show or alter it, it would be a lot quicker if you have the code that renders it. If you are in the business of writing papers that you want people to read and quote, it seems like a good idea to master the tools that help you do that.

https://www.lyx.org/

LyX is salvation. Edit in WYSIWYG, export to latex or PDF.

Good LaTeX is less about aesthetics and more about having boilerplate code and macros which greatly simplify the process of making readable documents with citations in the correct (nitpicky, archaic, utterly idiosyncratic) format and figures correctly included and labeled and captioned and scaled.

The joy of LaTeX is that it handles the aesthetics for you. You can focus entirely on structure. Or that's how it works when you have good boilerplate to start from.

I think the problem is that many people who don't use LaTeX (and some who do) are so inept at typography that it actually impedes the reader's ability to use the document.

One LaTeX-agnostic example is when people abuse hyphens-like this-which makes it impossible to parse the sentence without reading it several times.

Well you're onto something there. I liked typography for its own sake long before it became affordable to do it on personal computers, so I suppose I take my competence in that area for granted.

FYI, the .tex source is available for all papers uploaded to the arXiv.

No, you can upload just the PDF if you want.

OK, correction: the .tex source is available for almost all papers uploaded to the arXiv (because they give a big warning/plea message to do so).

You can now view (& compile) a large selection of latex examples, templates and articles directly in the browser on platforms such as Overleaf[1].

I'm one of the founders, and we're also making it possible to directly submit projects from Overleaf to repositories such as the arXiv and bioRxiv, and to traditional journals, to help speed up the submission and publication process.[2]

[1] https://www.overleaf.com

[2] https://www.overleaf.com/tutorial

Also, (experimental) biologists tend not to use TeX/LaTeX. Typically they use Microsoft Word and a proprietary citation manager like Endnote. Typesetting is seen as something the journal deals with, not the author.

I was a biologist, but I used LaTeX: it was important to have parseable references in a flat file.

Many of biologists stick to word file, especially the review/communet function. It is often because the collaborators stick to word and you have to follow up with it.

back in the day when I did this, I ended up writing papers with biologists. All the biologists would be cc'd on an email with the doc as an attachment and a "token" was passed between people to determine doc ownership. The person would make their changes and reattach the new doc to the email and pass the token to the next person.

Ultimately, this would fall over as people accidentally made concurrent changes and it would be hard if not impossible to merge between two documents. Doc files were never designed to be put into a version control system with merge semantics. Some people use shared folders with locking but locking doesn't work well over file shares.

I'm not sure but I suspect now people use dropbox instead of email; it's not clear to me how they manage conflicts and merges.

I rather my format be a text-line-based non-rendered format that I can render down if need be, but use the full power of patch and diff to manage conflicts.

I wasn't defending LaTeX; I don't use it any more.

I am certainly aware that many scientists don't want to publish their code out of embarrassment. It was a surprise to me, but I've heard that feedback.

I investigated various alternatives after I found LaTeX too arcane, including DocBook. It's still not entirely clear what solution exists to the collection of problems I have (want to semantically mark up my paper, want it to display nicely regardless of screen, etc).

What about something like markdown plus some syntax for formulas? You can compile the markdown (via a detour to LaTeX perhaps) to something that looks nice.

Yeah, some form of enhanced markdown, with self-rendering papers on github (or whatever-hub or whatever-xiv) seems like a very practical solution. It couples the paper's representation to the version control system in a way that could be convenient and expressive.

I figure that any markdown dialect enhanced far enough to be usable would be as complex as LaTeX, only without the tooling and pre-written macros.

Enhanced versions of markdown are just turning it back into a full wiki language.

Markdown gets very limited very quickly.

Are you aware of pandoc?

http://pandoc.org/

Shame in 'science', I find that unbelievably absurd.

About the 2nd part, may I rephrase it as PDFs are the docker of publication .. (Adobe would probably be very proud of that).

Or, if published, we could develop tools to normalize the data and improve the state of the art.

This is the right approach. Like what OCR does for digitizing volumes of literature.

OCR for historical literature is a really hard problem. Beyond the fact that OCR quality is pretty low, perception of the meaning of the papers isn't addressed by that. So it makes it easier to access (can read papers on the net rather than going to a library) but that's about it.

> it makes it easier to access (can read papers on the net rather than going to a library) but that's about it.

I disagree. The text is indexed and searchable.

> the meaning of the papers isn't addressed by that.

I agree. I wonder if starting with research papers that have lots of symbolic logic (e.g. math proofs) would be the easiest starting point for a system like this.

HA! Not to mention unflattering comments in LaTeX or BibTeX files. You thought source code had expletives... yeah.

Some of my .tex files may or may not contain a lot of profanities.

> Do you realize that many researchers would be ashamed if others saw their latex code?

The exact same think could be said about HTML, but that hasn't stopped people...

Reply on news.ycombinator.com