Augmenting the Markdown Language for Great Python Graphical Interfaces | Hacker News Reader

Augmenting the Markdown Language for Great Python Graphical Interfaces

taipy.io

97 pointsnevodavid78 comments

The big problem with Markdown is that it is horribly underspecified. Everything beyond basic text formatting and hyperlinks behaves differently in each markdown dialect (e.g. tables). What is the benefit of bolting additional use cases like UI rendering onto this mess?

There is a fully specified markup language, asciidoc, that reads similar to markdown and provides everything you'd need (as it's equivalent to docbook).

Unfortunately, it's much less known and there is much less software support for it.

That's the reason why I have not used asciidoc, the libraries are either incomplete or inexistant for many languages.

Too bad because I found it superior to markdown for my uses.

The problem with AsciiDoc is that its only maintained implementation AsciiDoctor is written in Ruby, which is a nice server programming language, but not the most logical choice for a multi-platform desktop documentation tool. Sure, there is AsciiDoctorJ, which is a Java wrapper around it, but for PDF generation, UML Diagrams etc you land pretty quickly in Ruby land again, which can get complicated depending on your platform.

PlantUML works pretty good in Asciidoc? If you don't want to set up a path to the local jar, you could use Kroki if you don't mind using a service (or you could run Kroki locally).

But I get what you're saying.

There's lots of speedbumps setting up graphing renderers, and the non-Ruby PDF engines aren't optimal (asciidoctor-web-pdf, which is the Paged.JS/CSS PDF toolchain) or they're too arcane for casual users (the DocBook stuff, i.e. DocToolChain/DBXSL/FOPUB). As for the core processor, though, AsciidoctorJS is pacing more or less hand in hand with core Asciidoctor.

Speaking for myself, it was worth the hit, but I'm an edge case. I work primarily with hardware/manufacturing, so I needed the transclusion, conditionals, XML equivalence[1], decent tables, front matter/indices/running content, etc, all in one core spec, because it takes eight months to get a new piece of software. Running back and forth to get permission for every Markdown extension and variant was a non-starter :D

[1] To a nice "relatively sane" XML spec like DocBook

I think the target is Python programmers that don't know about UI dev. Markdown is one (potentially confusing, I admit) way to create simple output, with a trivial layout. One can create a Python variable, reference it with a pretty simple syntax and get the thing displayed on a web page in the blink of an eye. There's also a Visual Studio Code extension that provides a preview of the page if needed.

For a finer control of the rendering, HTML is still available.

Is this still true with the commonmark spec?

https://commonmark.org/

CommonMark is a very basic Markdown specification — simple inline formatting, headings, lists, code blocks. It's pretty much Gruber Markdown + fenced code blocks. Many documents need some more features, like tables, but CommonMark doesn't support them, and there are a few table styles in use in different Markdown implementations.

The major problem is having me visualise what is going to come out of the process

Every markdown-like language has some underlining and bullet points. I can handle that. Once we get into using graphviz / some other tool for drawing charts and graphs I simply lose it.

It is in the end much easier to knock up something in PowerPoint / Visio / paint tool than be clever with graphviz.

The only arguments against this are

1. Maintenance - and yes it’s a good argument. Changing the name of one box should be easy - but it is with some paint tool too.

2. Graphviz etc are much more “programmable”. Slapping together 6 boxes for your network diagram is easy. 150 is what graphviz is for.

The problem is that you stopped using markdown the minute you went “oh, 150 boxes with edges between them.” At that point you have to use “some other tool”.

Maybe that tool is taipy, or graphviz or he’ll LaTeX is great for this - but you need to master the tool then you get great results.

So things like this should be

“If you are already fluent with graph production in LaTeX, use this tool That links latex to markdown”

Otherwise spending a large chunk of your undergraduate thesis learning how to use the tool might not work so good.

Summary:

Just use dead simple plotly graphs in your sphinx docs. Something like “total number of words bar chart”.

Then it’s there and you can expand it as much as you can before you hit the complexity limit that is just simpler to grab a visual drawing tools and store a binary.

Every time I see people doing something with Markdown, I wish they just replace it with support for Djot[0] instead. It is a Markdown alternative by the creator of Pandoc and CommonMark that fixes all of the most egregious mistakes, while being legible and visually pleasant as-is. It is also syntactically similar to Markdown, which should ease adoption.

[0] https://github.com/jgm/djot

When the markup’s first design goal is to be parsable in linear time (and then proceeds to have a bunch of rules to support that goal instead of flexibility for humans) I don’t think there is much chance of catching on as a general markup language.

Markdown was originally a loose formalization of how people already wrote text only emails. It was how _people_ wrote, not how to optimize for the computer.

It's not like all humans are born knowing how to read and write Markdown. A standard that is slightly less flexible or intuitive for (some) humans while enabling vastly more machine applications could be a win.

Could be, but mostly isn't. This thread and others like it are always full of folks lamenting that some other markup language isn't as popular or well-supported or both, even though said language has more features, a better spec, and got there first.

Maybe Markdown just hit the sweet spot. But I actually suspect it's something else: as soon as it's very convenient to have machines ingesting and producing a language, it becomes tempting to create tooling that is unnecessarily strict in what it will accept while at the same time spitting out markup that is unusually difficult for humans to read. A sort of specialized violation of the Robustness Principle.

The most famous example of this was probably XHTML, but even HTML5 suffers from it. And so we end up with stuff like Markdown because HTML tooling is too hostile to work with directly, and AsciiDoc because DocBook XML is similarly awful. They don't have to be, but...

Sadly there doesn't seem to be a Python implementation.

Unfortunately, not all, ignoring the linebreak is one of the most

Not sure, but isn't using _ instead of the more intuitive /italics/ a big mistake on the syntax side?

Lots of things use underscore for italics, if you asked me to guess I'd certainly try it first (and asterisk for bold).

... in fact I just tried it and underscore apparently works in WhatsApp, TIL.

OTOH /italics/ reads to me as a regular expression, which ... is apparently not what you meant.

(as ever, "intuitive" varies wildly between people, and the trick is to pick the option that's intuitive for the highest possible percentage of your expected user base)

/ intuition is pretty simple: slanted mark for slanted text

What's that wild intuition of underscores???

Also, why would you ever read it as a regex in a human /text/ markup language?

At least in English, typographic conventions have historically equated underlining with italics, and the reason the underscore key is present on your keyboard is because it was used on typewriters to format text that would have been italicized in printed text.

Surrounding text with underscores to indicate italicization is intuitive to anyone who is familiar with that convention.

Personally, I find surrounding text with forward slashes exactly wrong for italicization, because I mentally apply a skew-transform to the text to make the slashes into vertical lines, which leaves the text itself slanted in the wrong direction. Backslashes would make more sense, and also avoid looking like regular expressions. But literally no one uses that convention and we do not need a new one.

I've described what intuition is, you haven't addressed it, familiarity with convention is not intuition, it's just knowledge. And given that convention is unintuitive (_ for underlining would be, and on typewriters underscores were also used to, you know, underscore text), that's a bad convention

Also, this /convention/ already exists and used, despite its low popularity (eg orgmode or Bear note app), so you're also literally wrong here

He said the convention that doesn't exist was -backslashes-.

Also, intuition is "understanding without conscious reasoning," so remembering a convention IMO counts.

Consider e.g. parental intuition of noticing subconsciously it's too quiet and immediately going to find out what the kids have done.

For UI, consider the floppy disk icon often used for 'save' - it's considered intuitive because the vast majority of users already recognise it without having to think about it, even if they've never had a system with a 3.5" drive themselves.

"Intuition is the ability to acquire knowledge, without recourse to conscious reasoning or needing an explanation" per wiki The convention is the precise rule that _ is italics, so no, you aquire no knowledge, you need to already possess that knowledge and remember that precise rule.

On the other hand, if you know that _ is __underline__, - is --strikethrough--, then you can acquire knowledge (guess) that / is /italics/ because the principle is the same - formatting text follows symbol's shape (though it's not a perfect mach since / transforms text, not adds new symbols unlike _ or - ). But that's still intuition (* for bold would not fit here)

The other bad analogies don't help resolve it, so best stick to the topic

From my point of view, you acquire the knowledge that this system also follows the convention by guessing that it does and trying it (though I was quoting a dictionary rather than a wiki; I usually do for word definitions).

I appreciate that you don't find the analogies convincing, but they do express how I (and many others) see the situation, and declaring it a failure to stick to the topic because it doesn't fit how -you- see things is unconstructive at best.

Underscores indicate emphasis, just like underlining would, but in text you usually get italics (I'd note that this is often an <em> tag so it being italics is a browser default style).

Plus underscores get used -lots- of places for italics; I've yet to see the / form in the wild.

A markup language being (as here) used for technical purposes can easily be interpreted according to geek writing conventions.

So the intuition for _ would be underlining! Pretty simple. Where would your intuition jump to for underlining in markdown if _ is italics?

Em is just another bad term - which of the emphasis styles do you mean?

Re popularity - so what? How does widespread bad turns into intuitive good?

Bear notes used / for italics, also Orgmode (is that geek enough for you?)

> Where would your intuition jump to for underlining in markdown if _ is italics?

You have you know the fact that actual underlining had been traditionally used as a substitute for italics in hand-written and typewritten papers.

As in, in a citation, when you give a title of a book, you underline it, unless using a word processor with fonts, in which case you use italics.

Learned this in high school.

Did they teach you in high school the relevance of that history? Does the fact that it'd been traditionally used as a substitute mean you can't use underlining now, so it should not event exist in markup?

It's pretty obvious that underlining is something that rarely appears in print. It is mainly an annotation device: something added to finished print, by hand.

When was the last time you saw underlines in a book that weren't added by a previous reader?

Underlines are bad typography that should be used sparingly, if ever.

It's even more obvious that we're not talking about print, but a markup language anyone can outside of book publishing. So why do you want to prevent a "previous reader" from using said annotation device for the little note he's creating/editing?

Sorry, how did we get here from explaining the intuition behind _..._ denoting italics?

We got here because you mentioned all the irrelevant stuff your learned in high school, and when I pointed that out, you mentioned some more, now about the whole underline, which I used as an example of how there is no intuition behind _ since that intuition is "taken" by underlines (that's why you needed to criticize underlines)

I second this. I discovered Djot last year and it's been my preferred markdown ever since. Very extensible, easily parsable.

Why not just HTML?

Too annoying to write by hand. also too annoying to read the raw text. Markdown is easy to write by hand and looks good enough without a viewer.

This is an eminently solvable problem. I've been writing my personal website in a variation of HTML for a while now which is perfectly easy to write and read.

  {main
   {h1 Title}
   {p
    Paragraph 1.
    {{a {href google.com}} Here is a link.}}
   {p
    Paragraph 2, now with {b bold} text.}}

The nice thing about HTML/XML is that it's a clear data-structure, so you can essentially read it in any format you like and convert that to the underlying tree structure without much issue. That's why it's been so successful, basically the same philosophy as a lisp. Why someone would look at that and think "let's go back to a document with no clear structure" is beyond me.

You solved it with s expressions.

Replace the braces with parens and you can parse that into a tree with only a handful of lisp macros.

The braces are quite deliberate. Firstly, parens are used often in writing and it would be a pain to have to escape them. Secondly, it integrates much easier with the Scheme code of my site generator this way. I can write in a file, something like:

  (define (code-block file-name)
    {pre {{code {class block}}
          #(cmd "highlight" (thisdir (string-append "posts/" file-name))
                "-O" "html" "--inline-css" "--fragment"
                "--line-numbers" "--line-number-length" "3")}})

And it fits quite nicely with surrounding code. The mapping of these curly expressions to Scheme ones is relatively simple, though there is a special case for joining words to brackets.

  {a {b} #c #@d} => `("a" ("b") ,c ,@d)
  {a {b}c} => `("a" ("b") join "c")

Sure, I didn't mean to imply that you didn't know what you were doing. I was just adding more context for those readers who don't know what an s-expression is.

FWIW, my first static blogging system (circa 2003) did almost exactly the same thing you are doing (except that I used `[]` for lists, not `{}`). IIRC, it was maybe 5-6 macros+functions (and some readtable magic) to turn every `[foo ...]` into `<foo> ... </foo>`, with support for attributes using `:name=value` syntax and a list of exceptions for non-closing tags such as `<br>`, etc.

I redid something like that recently, in C, and it turned out to be unreasonably larger to write than in Clisp.

TBH, these days it's better to simply stick to markdown, because the tooling is much better (previews as you type, tools to automatically generate the html, inline HTML for what markdown won't do, etc).

Apart from the p tag, that gets annoying when you have long stretches of prose, I find plain html5 much less annoying than markdown. Moreover, tables and hyperlinks are clearer and easier to type in html5 than in any markdown flavor.

Easier to type? Sure, but I don't think HTML5 tables are easier to read than markdown tables

Thanks to optional closing tags, html5 tables are quite neat:

  <table>
    <tr>
      <td>first row, first column
      <td>first row, second column
      <td>first row, third column
    <tr>
      <td>second row, first column
      <td>second row, second column
      <td>second row, third column
  </table>

Does it get any better in markdown?

I'll convert your example to GFM.

  | first row, first column  | first row, second column  | first row, third column  |
  | ------------------------ | ------------------------- | ------------------------ |
  | second row, first column | second row, second column | second row, third column |

To me, this is more readable than the HTML version.

It's not fun to manually reformat those tables for readability. If I add something to the `second row, second column`, I have to add extra spaces to the `first row, second column`, or it will look janky.

  | first row, first column  | first row, second column  | first row, third column  |
  | ------------------------ | ------------------------- | ------------------------ |
  | second row, first column | exceptionally long second row, that makes second column look worse | second row, third column |

The fact that you need to list columns in rows already makes it NOT a table

Both are diffult to read and work with.

both are difficult to work with since it's not the job of a data format, but table editing tooling on top of that, but the markdown tables that look like tables aren't difficult to read unlike HTML

It depends on the intent. Markdown is meant to be readable on its own. If you don't plan on reading Markdown files in some sort of plain text editor, then the benefit of Markdown is greatly diminished.

Also, no one has implemented a really good experience for writing HTML, not for webpages, but simple documents. It could be really easy to write HTML mostly by hand if an editor could dynamically convert Markdown-like syntax into HTML tags on the fly. That way, the resulting document is definitely HTML, and not up to the interpretation of some flavor of Markdown.

Such an editor can also do things to try and soft-preview what emphasis the HTML represents. Here's an example of an experiment I worked on in this direction a long time ago: https://codepen.io/Ravenstine/pen/gyMVNx

^^ Never mind the pompous attitude I had when I wrote the placeholder text for it.

Markdown lacks prompting for inputs and running javascript but is easier to write and offers a quicker update, preview, deploy cycle. Preview and deploy is often a simple save vs. a build and deploy process. By extending markdown to support prompting for inputs and running javascript, you close some of the gaps and get the best of both worlds.

Too verbose for some applications is one reason not

Why not just asciidoctor?

Why not XML?

The laundry list of problems that can land in one's lap going down the road of XML publishing is considerable. Many are non-negotiable, particularly on git-based version control or on modern computing environments with locked down Java versions. It would be boorish of me to cite all of them, one after the other, so try this exercise instead:

Using an XML specification of your choice, working from a simple desktop computer, and with no money for fifty thousand dollar "end-to-end publishing solutions", perform the following: publish a document that uses conditional processing to show two different outputs for two different audiences while using a shared set of document components.

<kitchen_timer/>

What did you end up using? Anything neat?

I'm going to bet you don't find anything at all.

The tooling in XML publishing is weird, locked down, horrible, or all three plus some. Plus a lot. So, even if we pretend XML publishing is the bee's knees, it loses the speed test, it can't work on vanilla computers/requires admin rights constantly, and new writers train up on Asciidoc, from zero, in four hours. At a glacial, defense-industry-standard-for-fifty-year-old pace. Git probably provokes more existential crises in training than the markup does. Nothing compared to the business side, of course, but this is the defense business, and it's all madness.

I haven't really looked into Python GUI stuff much since the old TKinter, WxPython, and Qt days (5-10 years lol). Every now and then I check if a decent table widget exists like C# has had forever, but it seems like I'm always disappointed as the ones I've found can't handle much data or don't have filtering or whatever. Has this changed yet?

Eventually, I settled on a workflow with Python as a scripting language, but I avoid any UI stuff. If I need good charting, I just build Tableau or Power BI displays to consume the Python outputs that I've already pushed to a SQL database for analysis anyway as the data I'm dealing with is generally fairly large (i.e. gigabytes were generated while I wrote this reply).

I certainly get the appeal of doing everything in one language and there are some efficiency gains for sure, but sometimes (especially dealing with data science folks), I feel like they'd be much better off if they could learn common unix command line tools and SQL and add that to their arsenal. They all say they know SQL, but can't do much outside of "Select * from table;" and then spend a ton of time working around local Python memory limitations, which sometimes just means using a tiny dataset that they can actually fit into memory. I got a little off topic there, but wonder if anyone else has experienced something similar?

I gave up a long time ago attempting to write GUIs in pure python. I've come back to them periodically to see if the landscape has changed, but I'm always disappointed.

My current go-to python GUI solution is to use the Eel library. It renders GUIs in a web-browser like window with HTMl/CSS/JS.It lets you expose your python functions to JavaScript, so data can be easily transferred back and forth between the python and the JavaScript. And since it's just web-dev, it's pretty easy to make things look and feel good enough. Any other web-dev libraries like bootstrap and jquery can be used. It works pretty well for writing GUIs that put data analysis tools in front of my colleagues.

https://github.com/python-eel/Eel

This is really cool. Unfortunately Markdown tables are quite limited, lacking concepts like filtering, sorting, row/column spans etc. Also, more flexibility around styling would be great. Maybe they can add attributes to the Markdown and support for a simple scripting language.

To show-off these "hyper-markdown" capabilities and avoid compatibility issues with "md" files, we could name the new file format with an "html" extension (HyperText Markdown Language).

Or, to emphasize that the new features provide a significant upgrade from Markdown, we could call it Markup.

You can use a Taipy visual element called table to sort your table, filter it, add styling, add an action when a row is selected, to edit it, to add a row or todelete a row

Looks cool !

Makes me think of Voila [0], in my opinion one of the most powerful solutions to build interactive app with python. It requires Jupyter notebook, IPywidgets and e.g. BQPlot.

[0] https://voila.readthedocs.io/en/stable/

I've started using asciidoc[0] and find it much more friendly because it is both better specified and flexible.

[0]: https://asciidoc.org/

I think it would be nicer if the tool worked in reverse; that is, rather than making a markdown string inside a python script, just run the python from markdown (similar to the approach r-markdown takes).

Then, the part where you do "Gui(page=my_page).run()" just becomes a build script that happens in the background, rather than requiring the coder to have a bunch of isolated markdown snippets throughout their code

Python Graphical interface apps are getting really good at their job. You've mentioned Plotly, Stream and now Taipy. Reflex is also another one.

I've been using MDX and really loving it. Mixing code and markup fluidly is really nice.

Was hoping this was the analog for Python, but it seems the content is still strings.

i.e. parse markdown with code vs parse code with special strings

Mobile version of the website is broken by image overflow

https://i.postimg.cc/05btvjQ5/IMG-1590.jpg

Is there a Python library that uses Markdown or another lightweight markup language like reStructuredText for TUIs?

https://asciidoc.org/

Every time you glue on an extension to your markup language - or do a specialization with DITA, or modify S1000D via the BREX, or combine a dozen different MIL-STD DTDs to write your snowflake publication - you've reduced the volume of the tools ecosystem needed to render and write the thing. If everyone does this, you're effectively using a bespoke doc spec for each application, or even for each program or product.

Not all computing platforms allow users to do whatever. Eventually these disparate doc platforms will need to get merged, updated. What happens when Restrictive Platform X needs to update your Special Format? They fork, and they use a doc markup that's workable on a generic platform. Welcome to Double Work Town.

Anyway, it's pretty important to select a markup whose core specification does all the things you need on a generic computing platform[1]. Many different paths branch from that one singular fact, but I'll stand by that singular fact. Seen too many decades flushed down the toilet of churning doc/markup formats.

[1] That doesn't require frickin admin access for trivial writing tasks, like changing DTDs.

<cough>Arbortext</cough>

How do you pronounce “taipy”

I'd guess as "tie pie"

Yes, this how we pronounce it at Taipy R&D !

The author didn’t mention the arguably most popular python frontend library: gradio

I've never heard of this and I've worked with Python for decades. Maybe it's only popular in certain corners of the ML world?

Taipy is 2 years old. It was developed not only for Data scientists but also with end users in mind. First, Taipy was created because we failed miserably when we tried Python GUI tools on real data/AI projects. We tried with tools like Streamlit or even with Gradio. These are ok for having a quick pilot for data scientists not for real projects. In addition, we wanted to embed the concept of scenarios in our backend to easily perform what-if analysis, compare your executions, etc. We can go into more details if you are interested to delve deeper into the advantages of Taipy: performance, support for large data in your gui objects, style kit customization, gui events, sync/async calls, etc.

...this really sounds like an ad.

I am not in the market for any of this stuff, but as a piece of advice HN readers respond much better to technical marketing than anything that smells like sales copy.

If you were you going to rewrite this, I would expect to understand WHY your thing is different or better. Does it render faster? Does it have a cleaner API? Is it built on different primitives?

"These are ok for having a quick pilot for data scientists not for real projects." <- this sentence doesn't mean anything.

Never heard of it either. "Build and share delightful machine learning apps, all in Python." I am wondering what the heck is so special about "machine learning apps" that would differentiate them from any other app that has a UI with a Python backend.

It is optimized to demonstrate models, to create interfaces for training, and to gather feedback, Interface is input -> model -> output. All in browser, has many good and easy to use components and you can let it run on Huggingface with a oneliner.

It’s pretty much the driving force behind all open source machine learning demos these days, it was acquired by hugging face a few years ago and they do an incredible job maintaining it

Every developer needs to learn two lessons about great interfaces:

1. Layout, format [aka style], and interactivity, is both hard and necessary

2. Interfaces are subtly complex

Number 2 is the reason that "simple" formatting markup, like Markdown, will never be good enough to make a great interface. You can make a poor interface with Markdown, and that poor interface is good enough for very simple use cases. But people quickly get annoyed by simple use cases. They create friction, both for the designer and the user. More useful functionality is desired, and you need something more complex to enable that.

I've written a lot of tech docs. Markdown is one of the worst methods I've found to write them. Not because its formatting isn't intuitive, easy to remember, and easy to use. But because the documents it creates are not conducive to the purpose of the document: making it easy to read and remember information.

As for more general interfaces, the easiest way to understand why you need more elaborate methods, is HTML. When HTML was created by Tim-Berners Lee ~1990, he intended for format and layout to be independent and configurable. However, for years it lacked both formatting and layout options, and lots of people complained. It took six years for CSS to eventually become a standard, and before that, lots of hacks and incompatibility (and it turned out, more of them afterwards). The entire time people were complaining about how any document editor could easily format and layout exactly what the author wanted, but web browsers persisted in being difficult and inferior. Tables were only added in HTML 2.0, five years after the Web's birth.

You can still get by with not-very-good markdown, for not-very-good interfaces. But you will never have a Great interface without the ability to control layout, format, and interactivity, in subtly complex ways.

There is no easy way for a human to do that with text. But there is an invention that enabled users to create complex and rich layout, formatting, and interactivity: the WYSIWYG interface. With this tool, the human no longer needs to remember control characters, markup syntax, or formatting codes. The user no longer needs to spend endless time typing in code, displaying the result, being unsatisfied, and tweaking the code again, trying desperately to just get the screen to show them what they know they want. A WYSIWYG editor enables a human to completely discard any knowledge of how to format or layout content, and instead simply move things around on their screen with a mouse, completely eliminating complexity for the human, and enabling the creation of better content.

I think the reason modern software abandoned WYSIWYG, is that modern software isn't created for users. Most modern software is open-source (or close), and as such, it's created by developers, for developers. Developers don't want to spend time engineering a WYSIWYG, because they have no need for a good interface, and they don't care about wasting their time writing code. To them, writing code is the whole point, regardless of how long it takes, or what the outcome is. That's why Markdown was invented, and why people persist in trying to force Markdown to make something better, without realizing that that's impossible. They want to to build a castle, but they don't want to use anything other than the twigs and rocks lying about their feet.

That’s an advertisement. And the idea isn’t really groundbreaking or novel, just some custom tags in Markdown that can run arbitrary Python code and offer some basic two-way interactivity.

Agreed, plus I must imagine that Python's opinion on syntax and whitespace may not make this a most ergonomic solution.

Where does this verge into the realm of a Jupyter notebook? And why would this be better than such a notebook? The usecase for this markdown and rendering is... vs. a notebook? Genuinely curious about this, because sure you can make code do stuff, but is it worth it?

Reply on news.ycombinator.com