Madoko – Write full-blown academic articles in Markdown
madoko.net
madoko.net
It supports both Markdown and AsciiDoc and is used by authors to write books, documentation, research-papers (GitBook has great TeX support, in all outputs: web, pdf, epub, mobi). (Here's a math heavy book for example: http://jandeleeuw.gitbooks.io/bras1/content/blockrelaxation/...)
The format and toolchain itself is open-source (https://github.com/GitbookIO/gitbook)
There's a ecosystem built on top of that such as a Desktop Editor (https://www.gitbook.com/editor), plugins (http://plugins.gitbook.com/), and more ...
I don't want to hijack the current thread, Madoko seems really cool, I thought some of you might be interested in a more established solution. I'm happy to answer any questions !
Disclaimer: I'm one of the GitBook co-founders
Still not finished it, since I wanted to make it realtime collaborative.
You probably want to update your website in that case: "Markdown, books are written using the markdown or AsciiDoc syntax. TeX support is planned."
The website mentions TeX but means LaTeX, I'll get that fixed, sorry for the confusion.
Our TeX support allows you basically to write Markdown/Asciidoc and use TeX formulas wherever you need to, which gives you the best of both worlds (the simplicity of Markdown and the power of TeX formulas).
Your website talks about GitHub. Is it specific to GitHub or does it work with any Git repository?
What if you shut down? How do I continue to produce my book/paper?
We have great GitHub integration because it's the main Git host out there, we store git repos too.
If we shutdown, our toolchain and book format is already open source (https://github.com/GitbookIO/gitbook), so no matter what you'll always be able to use that to build your book locally. Secondly our Desktop Editor (https://www.gitbook.com/editor) works offline so you could even continue to use that. And well since your book is a git repo, you can have a local copy, a copy on GitHub or any other git server of your choice. So I think you should be good :)
We don't believe in vendor lock, we're big supporters of open-source and we want users to use our tools and platform not because they're obliged to, but simply because it's the best workflow/product for them.
Does that make sense ?
Do you have any support for reviewing, in a way similar to Word's 'Review' (comments and tracking addition deletions)? Using version control and branches is not OK, because non-software people cannot wrap their heads around it (just getting them to write in anything but Word is already a big enough hurdle...)
Do you have a proposed workflow to work with formulas? Inline TeX (or LaTeX) is again a no-go, it's too finicky to get right for people who don't have Edit/Compile/Preview Stockholm syndrome (bitter? me? nah). What I'd settle for is a separate file for each formula, and a graphical editor that can edit those files. A somewhat convenient desktop-based editor that'd give an overview of everything would be better (so having some way to start EqualX from your desktop tool would be enough).
What experience do you have with Gitbook to deem that it doesn't add enough value to deserve the payment?
Discussed here on HN previously: (https://news.ycombinator.com/item?id=10030585)
This reinforces some ideas that have been percolating in my brain for about two years. Between Gibook, PollenPub, and now Madoko it is now apparent that for the first time in its existence LaTeX is getting some competition.
What LaTeX gets right is that it produces beautiful structured academic-type (references and figures and such) documents. What it gets wrong as Gruber's Markdown has shown us is that the basic units of text should look like text and trigger markup through whitespace and intuitive but ultra-minimal markup such that without a processor it looks like an anally retentive writer produced the document. It codifies ascii practices. It's one of those ideas that until it happens you'd never think of it but afterwards you're going, "of course!". Besides Python and Markdown, do any file formats work this way?
I'm not going to make a Steve Yegge type predication but if I were I'd say that some form of scholarly markdown with multiple toolchain implementations (with git integration) is going to oust LaTeX as the structured document tool of choice. Be interested in what others have to say on the matter.
TeX and friends has adapted over the years through external tooling, without changes to the core, for example moving from postscript to pdf, the new engines XeLaTeX and LuaTeX. Madoko is just a preprocessor for LaTeX. If you had to finally do some changes to the LaTeX code it produces it will be very difficult as this LaTeX code is not very clean. Having said this though Madoko and similar projects are great for the LaTeX community, as an entry point to newcomers and to occassional users. It is a great Project especially as is written in Koka in itself a beautiful Project.
Example config for latex export (although I do not write academic articles):
https://github.com/cadadr/emacs.d/blob/c81ff8afa3ca4ab553013...
I learned the hard way. I wrote some presentations using org-mode. Then they changed the exporters without updating the documentation. It would be especially problematic to write an academic article in org-mode, because you might have to return a year or two later, and that's not when you want to find out how much has changed since you last worked on the paper.
One of the best features of this pipeline is the fact that I can write a large chunk of the document whilst conducting research. I can jot down notes, caveats, schedule TODO items etc. and just tag the bits I want for my own reference as 'noexport', so that they don't make it into the exported tex or pdf document.
Another great feature is that I can embed code for producing tables/figures all within the document and specify whether I want the code or result or both to be included.
Finally, once I am happy I have most of the document the way I want it, I export to tex to deal with the final tweaks to finesse things -- spacing for the entire document, fine-grained adjustment of figure formatting.
This ensures that I have all the information that went into constructing the document and allows me to deal with various venue-specific constraints like submitting a source archive, or pdf as necessary.
Here are some resources for getting started with this:
- https://github.com/vikasrawal/orgpaper/blob/master/orgpapers...
- http://draketo.de/english/emacs/writing-papers-in-org-mode-a...
- http://www.clarkdonley.com/blog/2014-10-26-org-mode-and-writ...
http://research.microsoft.com/en-us/um/people/daan/madoko/do...
http://webcache.googleusercontent.com/search?q=cache:azgHiPZ...
Could someone correct it?
For me a strict requirement for this is that I can use a particular template and document class. Unfortunately I don't have any choice as to what the particular format is. It's not clear if Madoko makes this possible.
It would be nice if journals would accept Markdown, but frankly I don't expect that to happen anytime soon.
Perhaps it would be easier to go the other way -- to modify a Markdown parser to output a simple subset of Latex that can fit into most common academic templates.
As far as I know, a lot of the journals have their own automated build systems for the manuscripts to perfectly fit into their designs.
Nevertheless, I've also encountered PDF uploads. But even then, the PDF will be automatically checked for consistency.
[OTOH, I've seen claims many publishers already have a process for publishing Word submissions — something involving monks spending their lives typesetting everything from scratch? — and so when they do get a nice LaTeX, they throw the source out re-use the above process! Perhaps that was only about the bibliography though.]
I've seen some Springer (LNCS) submissions, and those definitely included the .tex source.
arXiv requires the source, mainly so they can re-compile them in the future with better tools [https://arxiv.org/help/faq/whytex]. Unusually, they make sources available for anyone to download.
I'm one of the founders of Overleaf [1], and we've taken the route of keeping the full power of LaTeX there for anyone that needs it, but providing a 'Rich Text mode' [2] overlay for those who prefer a less code-like experience. If you give it a try, let us know what you think!
(Note - we work in the same office as figshare, so this is probably a biased recommendation!)
I actually had quite a lot of fun figuring out how to get everything working in the cloud. Set up my home desktop as my personal computer "cluster" (i5 processor, so a cluster of 4 cores!), I find it hard to work in an environment that I tend to do other stuff besides work in. Derived a small representative dataset from the massive one I had to quickly iterate over my analysis scripts in Cloud9 (while having github take care of my revision history). Then ssh'ing into my desktop PC whenever I needed to run the scripts on the full data set.
I considered adding a section on open science and cloud-based software to my thesis, but in the end decided not to due to a lack of time. Open science is based on open source principles, where everything that can be shared should be shared. The Cloud VM angle is quite interesting in this sense, as being able to share the machines that software for a research project was ran on is often quite difficult (or nigh impossible), and a cloud-based VM seems to bridge this gap somewhat. In theory, anyone could log-in to the VM and play around in the workspace I set up for my thesis project, not that many would want to I suppose.
Also, there's this presentation: http://research.microsoft.com/apps/video/default.aspx?id=238...
- Pelican - static website generator, like Jekyll but in python.
- MathJax to render Latex
It is easy to synchronize Pelican with dropbox or github, and it is incredibly convenient.
You can use this article:
http://technivore.org/posts/2014/01/03/blogging-with-dropbox...
To configure it in such a way that all the updates happen automatically.
For example I can simply write an article in Editorial on my iPad, and it gets immediately published/updated, without me having to do anything at all, exactly zero mouse clicks.
https://github.com/PeerJ/paper-now https://peerj.github.io/paper-now/
My only complaint is that I'd imagine reading academic articles in an ultra-thin sans-serif font might get tiring after a while.
Notably missing is math support, however, because rather than Madoko's approach of using LaTeX and then using a LaTeX-to-HTML renderer, asciidoc renders directly to HTML (or Docbook, or LaTeX). IMO, the end result doesn't necessarily look like an academic article placed on an HTML page… But it's nicer for consumption in a browser—the whole asciidoctor site is generated from asciidoctor.
EDIT: this means asciidoc is a better fit for documentation, in book or article or whatever form—but I think once it gains math support (which they're working on) it would be quite useful for this as well.
I would highly recommend asciidoc over markdown for scholarly work in succint plain text. Markdown is limited in its capabilities by design, as evidenced by all the different flavors of Markdown extension. It's what I would use to write an e-mail or blog post. Asciidoc, in the design of the language, anticipated extension and post-processing and has better support for using it as part of a complete document processing toolchain. After all, it's just syntax sugar for DocBook XML.
I'm not sure if this is a joke I'm missing, but "hiccups" is the spelling of the term referring to involuntary inhalation as well as (informally) a minor difficulty or setback[0]. A "hick" is an unsophisticated person[1].
What if you need to futz with margins and styles or use a publisher's standard .sty file?
It seems like this would be essential for writing research papers according to a conference's template. If so, I would definitely use....
You could modify the default template to use whatever style file you want.
Pandoc also supports references with a .bib file, and inserting TeX math formulae between dollar signs.
I've just been hoping that the pandoc world would progress (add more scholarly features) so that I could continue to keep my own data locally, manage it in my own vcs, and process it with a makefile.
The only reason I see to hop on the software as a service bandwagon is for collaborative real-time editing (and some projects benefit greatly from this). I'll plug sharelatex for that, primarily because their model releases the software so that you can run your own if you don't want to pay for the service.
On the Markdown+LaTeX note, I wrote my own Markdown parser (https://github.com/etherealmachine/markdown) to drive my github.io site (https://etherealmachine.github.io). Basically it's what you describe - easy mixing of Markdown+LaTeX to construct nice looking HTML.
http://research.microsoft.com/en-us/um/people/daan/madoko/do...
TeX embedded in Markdown seems painful.
How about writing in a restricted subset of HTML, as some forums (notably Slashdot) allow? <h1>, <h2>, .., <title>, <p>, <i>, <b>, <blockquote>, and a few others let you do what Markdown does. For math, there's MathML: Square root of two: <math><msqrt><mn>2</mn></msqrt></math>
You mean like ms word or open office ?
Markdown is really not sufficient, this markup looks very cool but I'm missing compatibility again.
I'll see if I can make it yellow cms compatible when exporting html...
> Madoko is fully compatible with basic Markdown syntax and passes the entire test suite. It also implements most extensions, like Github flavored markdown, PanDoc, Markdown Extra, and multi-markdown, and it adds quite a few features itself to make it really useful for writing academic and industrial documents.
Process is md (vim/mou) -> cmark -> wkhtml2pdf -> pdf
This works well for us, but would be nice if we could sort of merge this with an online wiki of some sorts.
Does it make a new push to GitHub every 30 seconds?