Polymath: A Markup Language for Everything | Hacker News Reader

Polymath: A Markup Language for Everything

jwmza.com

109 pointsjohnmorrison57 comments

Seems like a good idea, however I think it takes Markdown only a step in the right direction, without going the full way — allowing the user to specify custom "tags" and allow for custom styling of those tags (with html/css or some user friendlier way).

Basically, LaTeX, but:

- web ready, no pandoc tex→html mangling

- user friendly, allowing for 0 customization and still giving nice results

- with nicer markdown-like syntax for the common things, and a non-obtrusive way to include custom tags

- with an easier/better language behind it (not just macro expander)

I'm still just a student, but I frequently find the need to use something other than just math/list/image/link, e.g. theorem&proof blocks, Q/A blocks, you name it. The thing I'm currently using is Pollen [1], but it doesn't come with common tags defined and so the initial setup is a bit of a hassle.

[1]: http://pollenpub.com

Thanks!

Sorry, from your wording I'm not sure whether you are suggesting custom tags or saying there already are, but to clear up any confusion, there will be.

That's kind of what I was hinting towards at the end of the demo - you can define custom tags and styling behavior (either globally or at a block level, or site-wide in your preferences) with keywords like "@"

Trust me, I know the struggle of setting up like 500 custom math and environment macros in LaTeX. I'll try to find the best solution for this sort of thing - Polymath should work very well out of the box, but it should also allow for a fair amount of customization.

Thanks for the reply! Yeah, I was implying that I'll miss custom tags, so I'm glad to hear they are planned!

I was planning to do something similar myself (note-taking/blog-making system with good defaults and customization in a sane language), but you beat me to it. I'm looking forward to new updates!

This looks nice!

I and many many others have been using org-mode for years for these sorts of things; specifically the easy LaTeX integration and code highlighting, but there’s a kitchen sink worth of goodies that the user base has added over the years.

Besides not requiring emacs, is there a value-add with Polymath? Or is this mostly scratching your own itches — which is a fine thing to do but a different one?

Thankyou :)

I actually haven't tried org-mode but I've been hearing about it for a while now. Can't comment.

> Besides not requiring emacs, is there a value-add with Polymath? Or is this mostly scratching your own itches — which is a fine thing to do but a different one?

I don't have a concise and perfect answer for this yet, as it is basically just a very early version of a personal project I've been itching to work on, but these are what I think I'd like to do:

- The vanilla implementation of Polymath as an SSG should be extremely easy to learn - even easier than doing a pure HTML (+CSS?) site. The reason why is because I see a lot of professors with really great content but their page is basically pure old HTML and looks absolutely awful on a modern computer or smartphone. I want something that supports 99% of the (static) features most people want right out of the box: video support, great typography and layout for all devices, interoperability with most of the other languages or formats they already use, and a very easy learning curve. It should basically be possible to have only a bunch of Polymath files and render to a beautiful static website without needing any other technical know-how.

- Super long term focus: I want to make this the right way now and then be able to use it for the rest of my life, so it will be done from a perspective outside of the status-quo technology, as in, I want to think about the best way to concisely format a specific kind of information, in a way that is easy to render out to any current or future format (PDF, HTML, LaTeX, ...) that way anybody who builds up a considerable base of Polymath files should have minimal friction in adopting future technologies and swapping their infrastructure.

- Support other kinds of pages and layouts, like for example https://jwmza.com/portfolio/switzerland

- Something maybe a bit crazy: I have this idea to read TeX or tables and automatically generate interactive graphs, which I think could be really useful for people who do a lot of STEM writing. Say you write {x f(x):x\mapsto x^2}, I'd like to automatically guess the relevant domain, codomain and bounds, and create a little interactive graph.

You mention supporting graphs of functions. I use DOT [1] to render graphs of nodes and edges very often. Do you plan to support that type of graph as part of everything?

[1] https://en.wikipedia.org/wiki/DOT_(graph_description_languag...

Thanks for the link!

In my example I was referring to the Cartesian plane graph of x vs. f(x), but I do certainly hope to support the kind of general graphs / directed graph structures you're referring to. If there are open source libraries to process those kinds of things, I may be able to hack MVP-type usability quicker than otherwise.

Unfortunately Polymath is a side project that only has a limited resource pool available at the moment, as I have other work responsibilities.

I'm considering the options available to me in finding some way to fund a dedicated long term development of Polymath. Maybe a paid hosting service for ready-to-go blogs or something like that, or a Patreon-type deal if it's better to just keep the whole thing open source. Very early in the project right now - definitely open to any suggestions on expanding it.

Mathjax supports a latex graph library, AFAIK Katex does not. It would be nice to have tikz or at least a subset of tikz.

I'm open to MathJax or KaTeX or anything, nothing is set in stone.

> The only third-party components so far are the KaTeX\KaTeXKATE X rendering engine which generates beautiful and insanely performant HTML/CSS components from TeX\TeXTE X expressions – it is much, much faster than MathJax, which seems to be by far the most common

But is it as fast as running MathJax pages through mathjax-node-page to create static pages which don't need JS at all?

> But is it as fast as running MathJax pages through mathjax-node-page to create static pages which don't need JS at all?

Not sure, I should note that I've only ever used MathJax and KaTeX with the client-side JS rendering option, primarily because it's just faster to hack onto an early prototype.

If you prefer to do server-side rendering to avoid scripts, I imagine that the end user performance is roughly the same, however it is my understanding that MathJax has some advantages here in (a) also supporting MathML and (b) rendering to PNG, SVG, etc, while I believe KaTeX only has one rendering option in fairly CSS/HTML heavy nested sets of HTML elements and an imported web font.

Thereby I could see a potential preference for MathJax (esp. to render just a few PNG/SVG equations into the static site and serve those) but I think for really long documents, pre-rendered KaTeX might still be superior because you only need to load a single web font and the HTML itself instead of numerous copies of the same symbols in hundreds of image files. KaTeX also seems much faster if you do client-side rendering with JS.

Btw, your site has been a big source of inspiration to improve my own (in both design and infrastructure). I liked your post about A/B testing CSS and HTML, and the margin footnotes are really cool.

> Btw, your site has been a big source of inspiration to improve my own (in both design and infrastructure). I liked your post about A/B testing CSS and HTML, and the margin footnotes are really cool.

Thanks.

You don’t need JavaScript for KaTeX. So I suppose the two options are equally fast.

Apparently he does:

" <script defer src='https://cdn.jsdelivr.net/npm/katex@0.11.1/dist/katex.min.js' integrity='sha384-y23I5Q6l+B6vatafAwxRu/0oK/79VlbSz7Q9aiSZUvyWYIYsd+qj+o24G5ZU2zJz' crossorigin='anonymous'></script>"

Yup. For my personal site, I'll probably move the rendering server side soon if it doesn't add too much to the render time. Likely will make both options accessible to other Polymath users - I think server side and client side both have their merits.

I am using Jekyll with KaTeX and am reasonably content. The only thing I’m missing is a hyperlink checker. Since the whole site is static, Jekyll could check all the links to all the pages (even the ones to images or external sites). But it doesn’t.

I never understand why static site generators expect me to write the real link in my content, rather than a placeholder e.g. tofile:foo/bar.md which the SSG replaces with the permalink.

Antora's cross-reference feature[1] might be what you're looking for. It also has the very, very useful concept of content catalogs; it's an SSG for documentation sites.

[1] https://docs.antora.org/antora/2.2/asciidoc/page-to-page-xre...

Apropos "[...] for Everything", https://github.com/kisonecat/tikzjax might be of interest. It uses a WebAssembly copy of tex.

Regrettably neither KaTeX nor MathJax support category theory diagrams. And I'd like to try making interactive diagrams, to explore using mouseover supplements to reduce the learner cognitive burden of keeping track of all the moving pieces and their significance. Perhaps they might be combined with a generic js diagram library... but it doesn't look like anyone has bushwacked that path.

Thanks for the link!

I haven't used TikZ too much, maybe only a dozen or so times, but definitely am hoping to have something great for interactive diagrams. Will certainly keep this in mind as I add more features.

I'd be interested in having a look at what are the most important and widespread types of diagrams and data visualizations in academic work and research, as I'd like to make that sort of thing better. I'm particularly inspired by the interactive visualizations on https://distill.pub , however I think those are painstakingly hand-coded in SVG+JS+CSS+HTML.

I’m trying to set up a chat system that would enable category theory diagrams. It seems like webAssembly is the only way. I’ve come across this library before, probably will take the plunge and try to get it to work in a larger chat system.

https://github.com/yishn/jsx-tikzcd and https://github.com/yishn/tikzcd-editor look interesting.

http://bl.ocks.org/larsenmtl/86077bddc91c3de8d3db6a53216b2f4... is an example of using d3 with MathJax.

I have a basic question partly related to the project. When a markup can be compiled so fast, what's the fundamental reason we don't have WSYIWYG software that saves in the markup format as the norm.

I'm sure that could be done but I'm not sure if it's the right thing to do. I think it's important for people to understand the syntax and what they're using.

I think it'd be cool to do a side-by-side web app where you can type your Polymath code and see the resulting page at the same time.

Think about how many people in the world are missing on the benefits of such markup languages because they aren't interested in non-WYSIWYG/M. The market potential could be much higher if syntax learning can be avoided.

It really depends on the specific user and why they need the language.

Ultimately, Polymath is meant to provide a file format that will work for the long term, so I don't think users should be always getting an abstraction of the language through WYSIWYG, because then they don't understand the files they are preserving.

If you only care about having a nice blog, then sure, WYSIWYG might be convenient, but the whole reason for using Polymath instead of pure HTML/CSS + JS libraries is because it gives you the same/similar level of convenience as a WYSIWYG editor without losing exactness and understanding.

I follow that viewpoint and am OK with the direction being taken.

On the other hand, even after asking many, I have never received a good answer to why the exactness and understanding cannot be obtained with a WSYIWYG/M markup editor.

They abstract away from the underlying code.

You wouldn't be able to tell the difference between

  {b Bolded} {b text}

and

  b Bolded text

this becomes especially concerning with TeX, custom styling, etc

That makes sense, and I can imagine that.

However, should it not be possible for the software to find a minimal cover for the formatting that would produce the same visual output, thereby practically recovering what hard-crafted markup would have been?

Thanks.

Yeah, if I'm interpreting your words correctly, this would be like turning

  p {b This} {b text is} {b bolded} this isn't

into

  p {b This text is bolded} this isn't

But then the issue is as follows:

In Polymath, I'm planning on allowing basically arbitrary CSS styling by parent, id, class, etc. So if the user later wants to do something like put a rounded box around each span of bolded text, those two, while initially looking identical in WYSIWYG, end up completely different.

I can take that. Thanks.

The syntax and features remind me of Pug[1]. I find it's very nice to look at, but the significant indentation means you really have to have editor support for writing lengthy text blocks, without having to constantly manually add/remove whitespace. That's an advantage of Markdown, you can write it as easily in a textbox than a dedicated editor.

[1]: https://www.npmjs.com/package/pug#syntax

Thanks, and actually, you should be fine with just a text editor.

If the text editor doesn't support line folding spaced indentation, you can fall back to tabs, and if it doesn't support any indentation, you can actually write a paragraph all on one line. The manual wrapping is completely optional, I just like the way it makes my source code look.

Worst case scenario, you can always wrap any block in {brackets} and completely forget about all that indentation and newline stuff :)

Hi, just a heads up that I already have a FOSS project that might be a direct competitor: https://github.com/zyrolasting/polyglot

Not here to talk about which is better. I am excited for you and your project in any case since I understand the problem you are working on. But if you want to monetize this, you might want to be aware of Polyglot.

Okay, I've had a look at the project, and it seems to me like these are two fundamentally different things solving two different problems.

Doesn't seem like a direct competitor - although the names are 60% similar.

Looks excellent. One thing I'd question is why it should deviate from html / css like it does - why choose corners: 4px over border-radius: 4px?

Thanks! I've actually been putting some thought into that exact question with a lot of things (i.e. using Markdown-style identifiers for things like Headings [#, ##, ...])

I think what I'll probably do is support both the raw CSS/HTML keywords as well as a subset of more human-readable keywords and presets (my gripe with border-radius is that more often than not when I use border-radius, I don't even have any borders, what I really mean is corner radius)

Another example is box-shadow: I think the syntax for this CSS feature is horrible for people who don't use CSS all the time. I think some presets like `shadow: soft` would be nice in addition, where

  shadow: soft  |-->  box-shadow: 0px 2px 4px rgba(0,0,0,0.1)

Pretty cool! Seems like it would be a good fit for Jupyter notebooks? Data scientists might be interested in having something more powerful than Markdown

Definitely! In fact there are a lot of concepts from notebooks I've been borrowing for Polymath / other features on my website.

I'd certainly much prefer to use the Polymath syntax when using something like Jupyter (although Markdown has been pretty good thus far ngl)

The real advantage of Polymath will come once things like CSS grid features and block-style-macros are fully functional.

This is really good. I will plan to test it out tomorrow. You’ve solved some very real problems in an elegant way.

Thanks! Currently the MVP is basically hardwired into my personal backend but I've added an email subscription button which will notify you once I release a public/beta version. Still have a few things to fix up!

As an open source developer, I have to ask: Will this be released as open source, readily available on GitHub?

I'm not sure yet, especially considering that there are a few different components to the overall thing.

The language itself will almost definitely be open source, and I've created a MIT-licensed GitHub repo to support it as of earlier today.

The site generator, maybe a hosting service, and other such things might have a paid or freemium model to support the development of the language - but I'm not sure what to do yet. Might also just do some kind of donation backed fully open source thing.

Nothing wrong with trying to make money for your hard work. For the lurkers, here’s where the Polymath markup language GitHub repo may end up residing: https://github.com/jwmza/polymath

For my own site, I wrote a text-to-HTML converter in PHP to handle the basic kind of markup I need so things look better than just having simple HTML tags on a web page.

Nice, also look into AsciiDoc, which is a surprisingly well designed markup language for similar purposes.

Thanks for the reference!

I've had a look at a lot of examples and it seems great, but I really dislike the idea of homodirectional delimiters because I fear it unnecessarily complicates formatting and introduces a pointless risk of collision.

That's why I want to only use two kinds of delimiter:

- EOL, unless content is continued on the same indentation immediately without a gap, defines a block separation

- Strictly heterodirectional delimiters, e.g. () [] {} <>, define inline blocks and nested blocks, or can provide a bypass of the first rule

I feel that this design decision will save a lot of time, even though

  *italics*

is technically less characters than

  {i italics}

(however, in Polymath you can also just use a newline and save a character:)

  i italics

Cool project. Every date on your site seems to have an extra 1 appended to the start.

This is the Human Era (https://en.wikipedia.org/wiki/Holocene_calendar)

I believe it is a humbler, more appropriate calendar for talking about human history. I use it on my personal site because I don't need to conform to societal standards on my own site (although I don't use arguably superior numbering systems like dozenal or base-60 because that would be far too confusing)

I feel that Gregorian makes it pointlessly difficult to talk about anything between 10k BC and 0BC, and warps intuitive understanding of some of the most important time periods in human history - for example, ancient Rome sounds ancient, but understanding that it existed 10,000 years into human society and we are only 20% later is helpful.

Personally, I think the ~0BC Mediterranean societies carried out some of the most important and interesting work of humanity, and I'd like to improve my feel for the course of history around and since those times.

I'll probably do a blog post soon with an interactive map and calendar from 0 HE to 12,020 HE that really showcases the true time in between important historical periods.

I hadn't run across the Holocene calendar before. Interesting. I have a similar peeve around the diagrams showing the history of earth as a day and humans evolving in the last second or so, which makes our tiny sliver of time seem like "the end" as opposed to a tiny sliver on the way to the earth getting burned up by the aging sun.

This is a really great thought! I hadn't considered that before but what you're saying is really important for intuition as well, I think, for considering something like the Fermi paradox or great filter.

Perhaps not only is the Earth a rare planet with the right characteristics to support sentient life, but perhaps too we have evolved in a very tiny window between two much longer periods in Earth's life where something like a human cannot evolve.

It's always good to remember that where you are right now in your life is just in the middle of what will one day be a much longer past.

I signed up. How do I see how the markup language looks like?

I'll send an email to everyone on the list when there's a prototype you can play around with.

I've also made a GitHub repo where I might keep code and a wiki, or I may host a wiki written in Polymath on my site. It will explain all the intricacies of the language and people can do a custom implementation if they don't like mine.

As of right now, the demo page (this article's link) showcases the language itself in the preformatted text sections below or to the right of the document content (depending on whether you have a mobile or desktop screen)

The primary goals of the syntax are:

- Human readability

- Clear, explicit nesting

- Very easy learning curve

D'oh, I never noticed the right hand side. Thanks. The markup looks clean and usable

Is this supposed to be a non-free/non-open standard?

I basically started this thing very recently so idk what I'm doing yet, but the plan is for the language to be open source. I'm just trying to figure out what type of monetization I can potentially build around it (maybe a paid hosting service, maybe a donation type thing, idk) to help fund the development of the language.

But the language itself is open, it's just syntax, anybody can do whatever they like with it :)

Reply on news.ycombinator.com