Produce HTML from S-Expressions | Hacker News Reader

Produce HTML from S-Expressions

github.com

61 pointslelanthran61 comments

    (h1 :disabled Hello world!)
    (div :class="alert"
          So long, and (b thanks) for all the (em fish))

I hate that syntax for attributes.

:class="alert" is strictly worse than ((class . "alert")) or any other variation that actually leverages s-expressions instead of merging key and value into the same symbol. Ew.

Not a fan of the bare words instead of strings either, which means this cannot be naively evaluated by a Lisp.

I wouldn't say this is producing html from s-exps, rather that it is vaguely inspired by them.

> :class="alert" is strictly worse than ((class . "alert")) or any other variation that actually leverages s-expressions instead of merging key and value into the same symbol. Ew.

I hear you, but I wasn't aiming for leveraging the power of macros. Using `((class . "alert"))` does not provide any benefits over :class="alert".

> Not a fan of the bare words instead of strings either, which means this cannot be naively evaluated by a Lisp.

I traded off ergonomics against "Evaluation by a Lisp processor".Instead of

     (div "My first " (b "hello") (em " world"))

I'd rather do

     (div My first (b hello) (em world))

Thank you for your input.

> Using `((class . "alert"))` does not provide any benefits over :class="alert".

Of course it does. It is a s-exp, like everything else in this language, so it's just another list you can map, apply or otherwise transform. The keyword approach instead requires a weird incantation of

    (string-trim (cadr (string-split (keyword->string :class="alert") ?=)) ?")

to extract the value from the attribute. For what benefit? If the goal is using s-exps, why not use s-exps for everything?

> If the goal is using s-exps, why not use s-exps for everything?

My goal wasn't "Use s-expressions", it's "More readable and writable HTML trees."

This isn't a Lisp interpreter either, TBH. There's plenty of those around and they require the user to use a Lisp implementation, and write Lisp code.

map, string-trim, car, cadr etc are not required for s-expressions, they're required for Lisp.

If your goal wasn't "use s-expr" and it isn't a lisp interpreter, the tagline probably shouldn't be "Produce HTML from S-Expressions" and the project shouldn't be named "lisp-to-html"; it should be "Write HTML with parentheses".

> If your goal wasn't "use s-expr" and it isn't a lisp interpreter, the tagline probably shouldn't be "Produce HTML from S-Expressions" and the project shouldn't be named "lisp-to-html"; it should be "Write HTML with parentheses".

You're right maybe it shouldn't be called "Lisp-ish".

But, the sources really are s-expressions with an additional ':' marker for attributes, just not Lisp s-expressions with the ':' marker.

So "Write HTML with parenthesis" would be less accurate than "Emit HTML from S-expressions".

Who is your target audience? If it’s Lispers then you need to make the format macro-friendly, if it’s not Lispers then you need to figure out who else likes sexprs but not Lisp?

You could replace the = sign with a space, then you can have (a :href “/news”). Perhaps ::required for an empty attribute. This makes it better for macros.

> Who is your target audience?

It's not Lispers.

> If it’s Lispers then you need to make the format macro-friendly, if it’s not Lispers then you need to figure out who else likes sexprs but not Lisp?

Well, it depends. I like Lisp just fine, but all the Lisp ways of producing HTML are extremely unergonomic for those not already in the Lisp ecosystem.

Someone below posted a link to the clwiki page (I think, not sure) for a list of HTML-generation libraries. I've used some of these before, and they require the user to actually know Lisp well enough to program in Lisp.

I wanted something I can hand off to a junior who doesn't know any Lisp (and doesn't really want to), while still having something that makes it easier to type out HTML.

The `l2h` program is a compromise of sorts between ergonomics and flexibility. It's more effort to use actual Lisp, but you get more in return when you do. It's less effort to use `l2h`, but there are more limits when you do.

> You could replace the = sign with a space, then you can have (a :href “/news”). Perhaps ::required for an empty attribute. This makes it better for macros.

I made a note of this. You're correct about it being better for Lispy macros. Another responder elsethread pointed out a few things that could benefit the usage (imports, macros) while maybe not adding too much "cognitive overload" (AKA Too Much Lisp).

This project has only had about 6hrs thrown at it so far, so it is still early days.

I do appreciate your feedback. Thank you.

> Lisp ways of producing HTML are extremely unergonomic for those not already in the Lisp ecosystem.

Sure; pedaling a road bike is very unergonomic for someone who is jogging next to it. Makes sense.

> Sure; pedaling a road bike is very unergonomic for someone who is jogging next to it.

Exactly! There's no exhortation to turn joggers into bikers. Joggers on exercise forums aren't told to take up biking when they want to increase their distance.

It's exactly the same as asking developers to take up Lisp to use a non-Lisp tool.

i learned lisp, but i don't use it actively. so macros don't matter for me as much, but i have written a few parsers for s-expressions in other languages, most notably i have one in javascript. what is critical for that parser is that parentheses () do the grouping, and that whitespace is the separator between keywords and values.

with class=foo you introduce another separator, complicating the code needed to parse this. if you could use a space here, then i could use my already existing parser to convert this into json or even directly into html without needing any additional code to figure out class values.

simple s-expression parsers in many languages can be found here: https://rosettacode.org/wiki/S-expressions

To keep the rules simple (i.e. people shouldn't need to learn Lisp to write the source input) I don't want context to determine what `(foo :bar baz)` means.

With the current two rules, the symbol after any `(` is always the tagname. There are no exceptions based on context. The input `(foo (bar baz))` has meaning `<foo><bar>baz</bar></foo>`.

This meant I needed a different way to indicate attributes. I used the ':', resulting in needing a way to specify the optional attribute value, so I used the '='.

I still have to decide on a few more things, so I'm still considering changing the way attributes are specified, but I don't think I will ever make it contextual.

ok, that's reasonable. though you could specify attributes as (:foo bar) so that "(:" always starts an attribute. attributes without value could still be written without surrounding parentheses although always requiring them would make parsing easier.

> ok, that's reasonable. though you could specify attributes as (:foo bar) so that "(:" always starts an attribute. attributes without value could still be written without surrounding parentheses although always requiring them would make parsing easier.

There's been a lot of pushback on the attributes, so something there has got to change. I'm considering all the changes (including your suggestion too. Thanks, BTW).

The downside #1 with '(:' , from a usage perspective, is that '(' is no longer context-free and is inconsistent. For example, you could not produce a tree with `<:placeholder:>`[1].

Downside #2 is that it's just cumbersome to type (The whole point of this was to reduce the clunkiness of typing out HTML). Four attributes which would be

    :name1=value1 :name2=value2 :name3=value3 :name4=value4

turn into

    (:name1 value1) (:name2 value2) (:name3 value3) (:name4 value4)

I think if I adopt the reverse-smiley, I may as well adopt it in a way that is consistent - reserve all lists which has a ':' as the first symbol. Then, even though it's an exception to the general rule, it's a rule itself so users can easily abide by it and it opens up possibility for new functionality without breaking existing documents.

For example `(:attrs ...)`, and `(:import ...)` and `(:set-var ...)` etc.

[1] I don't know why a user would want this, but it's probable. After all, I use a similar syntax currently in LaTeX files so that sed can perform replacements before pdflatex generates my invoices using the transformed files.

'(' is no longer context-free and is inconsistent

i would not see it that way. "(" still starts a group, now it would just be either a tag or an attribute.

you could still parse the whole tree into a native data structure based on the parentheses alone (well, except for the significant spaces between tags, but that's another issue), and later look at the first element in each group to decide if it is a tag or an attribute

the same is true when writing. an attribute name starts with a colon ":foo". if you want to add a value to the attribute, you have to use "()" to group it with its name: "(:foo bar)"

another benefit is that you can write: (:class foo bar baz) instead of :class="foo bar baz"

If My and first just stand for themselves, you're not interpolating variables; where is the templating?

Also, what if you don't want a space between the bold hello and emphasized world? Or between first and <b>hello</b>?

> If My and first just stand for themselves, you're not interpolating variables; where is the templating?

Templating wasn't mentioned (yet. looking at the feedback I got, some avenues are open for some sort of templating).

> Also, what if you don't want a space between the bold hello and emphasized world?

Then you don't put one in?

     $ echo "(div My first (b hello)(em world)) " | ./l2h -s
     <div>My first <b>hello</b><em>world</em></div>

> Or between first and <b>hello</b>?

Again, don't put one in:

    echo "(div My first(b hello) (em world))" | ./l2h -s
    <div>My first<b>hello</b> <em>world</em></div>

I think that that is is intuitive as it can get for people who know HTML: space-emission matches what you would expect for HTML.

If you look at the git history, I spent a lot of time making sure that the parsing of whitespace and newlines made sense from an HTML perspective, not from a Lisp perspective.

That's not S-exprs then because there is no symbolic difference between )( and ) ( or between foo( and foo (.

But it's a better representation for HTML, that we would have had if Berners-Lee hadn't been a supreme goofball.

> That's not S-exprs then because there is no symbolic difference between )( and ) ( or between foo( and foo (.

You (and everyone else) who points this out is quite correct. I could have chosen a proper name, but when I started project (and created the name for it) I did not realise that I would have to make whitespace and newlines significant in order for it to make sense from an HTML perspective.

It was only when doing testing and trying to bold the middle two letters of "hello" that I knew I had to make whitespace significant.

So .. uh, sorry about the name :-(

This might be okay too?

  (div (class "alert") So long, and (b thanks) for all the (em fish))

  (div {class "alert"} So long, and (b thanks) for all the (em fish))

Yeah I think you want some way to clearly differentiate between tags and attributes as there are some cases where there are attribute and tags with the same name. E.g. does

(body (style "styleinfo")) mean

Or <body><style>styleinfo</style></body>?

second example using { } could imply the attribute and ( ) could imply the tag

> second example using { } could imply the attribute and ( ) could imply the tag

Right now the only reserved tokens are '(', ')' and ':'. Everything else is emitted as HTML content. Reserving '{' and '}' for use by the program means that users would have to escape '{' and '}' wherever it occurs in their content.

Personally, i'd go with

   (div (class [alert someothercss]) [so long and (b [thanks]) for all the (em [fish and chips])])

Like this amirite

https://github.com/weavejester/hiccup

The thi.ng umbrella has a hiccup variant for JS that looks pretty neat too: https://thi.ng/hiccup

Pollen's tagged xexp is probably the most sophisticated approach to this I've seen. Though hiccup is about as good and easier to work with. https://docs.racket-lang.org/txexpr/index.html

> Not a fan of the bare words instead of strings either, which means this cannot be naively evaluated by a Lisp.

Most Lisps (all that I know of) have a way to convert symbols to strings. The hard part is resolving the ambiguity between what should be text, and what should be treated as an HTML node. Like, what if I wanted to actually write the string "(b thanks)" on a page? R7RS could probably do this with the pipe identifier syntax[0], but I think R7RS is fairly unique in its capability for identifiers (and thus symbols) with such a wide range of supported characters.

[0] https://standards.scheme.org/corrected-r7rs/r7rs-Z-H-4.html#...

Bare words are fine until you have a variable with the same name as your bare word, so you need additional syntax to determine if you're referencing a variable, or it is just text.

> Bare words are fine until you have a variable with the same name as your bare word, so you need additional syntax to determine if you're referencing a variable, or it is just text.

This doesn't have variables. Or functions.

The space you embed it into might

you would normally quote the whole s-expression to make sure it is interpreted as data only. '(look at this: no variables)

To expand on this, Lisps also usually have a quasiquotation system to make it easy to embed data from the host system into an s-exp. `(look ,(get-parent): some variables like ,x)

> Not a fan of the bare words instead of strings either

It is consistent with HTML however, especially the fact that both collapse whitespace.

With this merged attribute syntax, since each key-value combination is a separate symbol and since symbols are interned, doesn't this mean that all the attribute values (and not just the keys) end up interned?

Do Lisp implementations usually garbage-collect interned symbols when they're no longer used?

Firstly, obviously yes for un-interned.

TXR Lisp:

  This is the TXR Lisp interactive listener of TXR 291.
  Quit with :quit or Ctrl-D on an empty line. Ctrl-X ? for cheatsheet.
  I'm confused; is this piano recital Rimsky-Korsakov or Wernicke–Korsakoff?
  1> (progn [finalize (gensym "foo-") prinl] nil)
  nil
  2> (sys:gc)
  #:foo-0012
  t
  3> (sys:gc)
  t

We make a gensym whose name starts with foo-. We register a GC finalizer for it which is the prinl function (print, with newline).

Then when we manually invoke GC, it gets collected; we see the print.

Internet symbols cannot be collected, normally, because they are always used: they are registered in their package.

TXR Lisp has weak packages, though. A weak package will purge a symbol that is not reachable other than through the package.

  1> (make-package "abc" t)  ;; t argument here means weak
  #<package: abc>
  2> (progn [finalize (intern "xyz" *1) prinl] nil)
  nil
  3> (sys:gc)
  abc:xyz
  t
  4> (package-symbols *1)
  nil
  5>

Don't know if other Lisp people thunk of this one, but there it is.

I don't think I exposed this weak property through the defpackage macro. It's intended for specific scenarios, like reading data in the context of a package.

The Scheme community seemed to standardize on Oleg Kiselyov's SXML format for XML and HTML: https://en.wikipedia.org/wiki/SXML

The initial driver for adoption was that Oleg made an excellent XML parser (SSAX) that used SXML. Then a bunch of people (including me) built new tools to work with SXML, and changed existing tools to use it.

Later, I realized that SXML permitting unnecessary list nesting, which initially seemed sloppy and inefficient, actually has a very useful efficiency property, once you're using immutable lists: you can compose larger XML with a small allocation and no mutation like:

    (list some-huge-tree another-huge-tree)

Racket-specific starting point: https://docs.racket-lang.org/sxml-intro/index.html

Hiccup syntax for Clojure uses hash maps (curly braces) for attrs, e.g. `{:style {:background "red" :margin "1em"}}`

See Reagent which uses Hiccup syntax: https://reagent-project.github.io/

    (defn simple-component []
      [:div
       [:p "I am a component!"]
       [:p.someclass
        "I have " [:strong "bold"]
        [:span {:style {:color "red"}} " and red "] "text."]])

A more interactive (but less examples) showcase : https://escherize.com/works/hiccup.space/

Also just because it’s not shown here, IDs can be declared with tag#id syntax.

FYI in traditional SGML DTD-driven HTML parsing, "disabled" in

   <div disabled>

represents the attribute value rather than the attribute name. The value has to be declared in an enumeration and must be different from all other enumerated values (of other elements) for the parser to infer the attribute and emit canonical markup such as

   <div disabled="disabled">

In HTML, it just so happens that the attribute name in what the WHATWG HTML spec calls a "Boolean attribute" is always the same as the single allowed value. However, WHATWG managed to break that in the latest spec, allowing "until-found" as additional value for the "hidden" attribute [1].

[1]: https://sgmljs.net/docs/html230116.html

If one uses the XML tree of the XHTML as the primary internal representation instead of S-expressions, no conversion is needed at all, as XHTML can be viewed in the browser simply by associating a CSS style sheet via "rel".

Nothing is a faster converter than avoiding a conversion step altogether... but that is not why I did it; I used that technique in my Ph.D. thesis back in 2007 [1,2] to avoid directories full with hundreds of auxiliary files (xml/ html.tmp/ ..), which may be confusing to manage, slow to copy, and eventually lead to the file system running out of i-nodes.

EDIT:

[1] https://www.sciencedirect.com/science/article/abs/pii/S01989...

[2] pp. 132-134 in https://era.ed.ac.uk/bitstream/handle/1842/1849/leidner-2007...

I actually really love this. I much prefer this over HTML syntax. Add a way to do custom tags and imports and this becomes a very, very neat static site generation tool.

> Add a way to do custom tags

I don't know what that means - it currently uses the first element of the list as the tagname, so you can use whatever tags you want to.

> and imports and this becomes a very, very neat static site generation tool.

I've toyed with the idea of adding more functionality (like imports, environment variables, conditionals, etc).

Not too sure I want to go in that direction.

The ability to create tag templates basically. You'd define tags that you could parameterize using attributes, and then within the custom tag "body" would be more tags definitions.

If the parser comes across a tag it doesn't know, it emits it verbatim. If it comes across a tag that is defined as a custom tag, then it replaces the tag expression with the expanded tag template body ("expanded" here meaning the parameters are substituted).

Similar to JSX, make an implicit parameter called "children" that holds all child elements, and then allow the templates to refer to attributes with a special expression (e.g. `(attr children)` or something, or maybe just `(:children)` if that doesn't break syntax rules).

> The ability to create tag templates basically. You'd define tags that you could parameterize using attributes, and then within the custom tag "body" would be more tags definitions.

Not sure how it might be supported in this library, but Hiccup (in the Clojure) ecosystem is similar and lets you do this with functions. This is a simple example of a scrollable div with a title block.

    (defelem scroll-column [ id title & contents ]
      [:div.scroll-column
       [:div.fixed title]
       [:div.scrollable { :id id :data-preserve-scroll "true" }
        contents]])

Note this (and Hiccup in general) heavily depends on Clojure array and hash literals. This wouldn't look as clean in a Lisp without those things.

> The ability to create tag templates basically. You'd define tags that you could parameterize using attributes, and then within the custom tag "body" would be more tags definitions.

This sounds like a good idea (could call it a `macro` but it might annoy readers who expect Lisp-type macros).

Something to DRY the input a little, maybe. I don't want it veering off too much into LispLand because then it's bound to be annoying to devs who wanted a more Lisp-conformant tool.

There are so many of these already: https://cliki.net/html%20generator

And still I can't find one that I like (which is why I also wrote one myself :D).

Reminds me of hyperscript like in https://mithril.js.org

  m("h1", {class: "title"}, "My first app")

Yep. Also it’s worth mentioning that that’s the signature of React.createElement

Here’s a simplified implementation: https://github.com/uxtely/js-utils/tree/main/react-create-el...

I'd like to thank everyone for their feedback and criticisms; I've received some excellent suggestions for improvement.

I've tried to engage with everyone - I apologise to those I didn't engage with or answer adequately.

Shameless plug of a similar idea to embed lisp into LaTeX files.

https://github.com/keyehzy/lispylatex

Nice, thanks for sharing. Curious what your motivation was for making it. Perhaps include a "why" at the start of the repo. As evidenced by some comments, depending on whether the goal was to write HTML with a lisp compatible syntax, vs, a more concise but simple xml alternative, it creates different expectations. Personally I'm just interested from a parser/ compiler perspective.

> Curious what your motivation was for making it. Perhaps include a "why" at the start of the repo. As evidenced by some comments, depending on whether the goal was to write HTML with a lisp compatible syntax, vs, a more concise but simple xml alternative, it creates different expectations.

You are quite correct, my goal appeared to be "Lisp interpreted HTML", my actual goal is "Write HTML tag trees easier".

If you use js-based hiccup instead of JSX, SPA frameworks would not need a compile step.

Is there a standard format for a-expressions?

I want to use them but I cannot find a spec like for JSON

I your aready using a scheme you can use SXML. The tooling makes XML fun.

> The tooling makes XML fun.

High praise indeed.

Well, it is because you can do most transformations using a tree fold. You get the speed and flexibility of SAX but with the nice interface of DOM parsing.

Apart from scheme, I have only done XML things in Python and C#, and that was a less than stellar experience.

Basically Haml then.

Haml is not homoiconic. It's like saying "the flask is basically a teapot". Sure, they both can serve similar purposes, but categorically they are very different things. They look different, they are usually made of different materials, and they are used very differently. So, no, it's not "Basically Haml"

Reply on news.ycombinator.com