Ask HN: Server-side HTML Templates based on DOM? | Hacker News Reader

Ask HN: Server-side HTML Templates based on DOM?

13 pointsPaulHoule23 comments

For side projects I've used BeautifulSoup in Python to break down HTML documents into the DOM, manipulate them at the DOM level, then serve them to a browser.

For instance the "main template" for a site could be an HTML page that has a <article> inside it which is a placeholder for the content. The content is itself an HTML page, and the <body> of the content is inserted into the <article>, also the metadata gets merged so the <title> passes through.

You might write

   <div>First Name: <span id="#insertFirstName"></span></div>

and the system would insert the firstName into that span, for instance.

I've done the same thing in Java with tools like jsoup and HTMLUnit.

This system can rewrite links and do other interesting transformations on the guest HTML.

One thing I'll concede is that this is slow compared to the alternatives. In a crude test a system like this and not optimized might be able to transform 10-50 pages per second on a core whereas it is easy to get 3000+ with string based templates.

I haven't seen other people do this, are there any examples you of know of DOM-based templating? Any thoughts?

If you consider static site generators as a special case of side-server rendering, then that is basically the principle that underlies my personal ssg [0].

Highlevel idea: It reads in all HTML files, parses them with JSDOM, builds a metamodel of the overall website, which (ab-)uses standard HTML meta tags as alternative to a frontmatter (e.g <meta name="template" content="articles"> declares the particular file as the template for the category articles, while <meta name="category" content="article"> declares a particular file as part of the category articles. And then it uses a non-standard HTML tag to call JS plugins with the whole meta-model of the site, so it can plug together content and template, rewrite parts of the DOM (for example add bidirectional links when desired, run a syntax highlighter on <code> tag, count words). When all plugins have run the DOM is serialized into HTML. Admittedly it could be a tad faster, but for my purposes it is good enough. I don't mind running a ton of JS on my machine, but still can serve mostly static HTML with very few sprinkles of client-side JS

[0] https://github.com/fnh/intertwingle/

Cloudflare has a thing called HTMLRewriter that is both a streaming HTML parser and selector for changing it. It's very efficient and let you change only the places you need.

For your example, you can just match in `[id^="insert"]` and add the actual firstName value in the handler.

I use it alot on the server side, but it's for JavaScript side, or Rust if you use the lower level lol-html.

The HTMLRewriter API is pleasant to work with. Bun has implemented it in addition to Cloudflare Workers. I would love for WinterCG to work on standardizing this API across JS runtimes so that I don't have to interact with lol-html across a WASM boundary in Node and Deno.

+1 to the lol_html HTML rewriter. Basically you can give it a bunch of selectors and code that will do whatever with what is matching. It's very fast and quite nice. It has bindings to Lua and a few more, but not to Python.

https://crates.io/crates/lol_html

I'm not quite sure what you're doing, but I wonder if a new-ish browser api - Declarative Shadow DOM - could be useful here. It allows for sending a sort of html shell to the browser that contains pre-defined placeholders, and then async sending the html fragments for those placeholders. No JS required.

It recently got support from all the major browsers.

- https://developer.chrome.com/docs/css-ui/declarative-shadow-...

- https://lamplightdev.com/blog/2024/01/10/streaming-html-out-...

I'm also considering trying something similar to you, and thought that perhaps GoQuery (essentially jQuery written in much faster Golang) could be useful here. https://github.com/PuerkitoBio/goquery

If you need to do the scraping stuff, Colly could be used (which uses GoQuery under the hood). https://go-colly.org/

You could also create a sort of hypermedia "single page app" if you serve the shell once, and then use something like HTMX (or even jquery) to swap the body and other html fragments without a full page reload.

Is this really different than Jinja2 or Twig with template inheritance and blocks ?

They want to be able to write logic to operate on the DOM elements rather than on Jinja2/Twig/etc template tags.

Twig has what it calls 'Live Components', which extend its HTML templating engine to update elements live over automatically linked AJAX calls. Basically turning its static templates into more of a react type system.

I think this sounds more like what the author means.

https://symfony.com/bundles/ux-live-component/current/index....

XSLT would be a classic answer, but it's a bit of a lost ancient technology these days. I've only used it in the browser for a lark, but I suppose you'd find mature (old) libraries for working with it easily enough.

PHPTAL is another system I've heard of (but never used) that seems to be built on this idea. It is also very mature (old). https://phptal.org/

In general working with DOM doesn't seem to be very cool these days. Everyone uses template systems that are glorified string concatenators, like Mustache, and doesn't care about security issues due to bad escaping.

I'm not sure what the advantage of this is vs any standard server-side templating (PHP, Twig, Mustache, Liquid, etc.)

Converting from a string to a DOM is very expensive because of all the non-XML non-standard cruft HTML accumulated over the years. If you can do a simple string find/replace instead, it should be less computationally expensive. No reason to actually parse the HTML unless you need to actually render it or at least work with it semantically/structurally, which doesn't sound like the case here.

How does this differ from using a templating engine where the server gets a template, and instantiates it to "the final thing it should be" on a per-request basis?

(also, if you need to transform 200 requests per second that's slowing things down, there's almost certainly a cache-layer missing?)

I made a web framework a long time ago that did this. https://twisted.org/documents/8.2.0/web/howto/woven.html

The work-in-progress Zine SSG project https://zine-ssg.io/ calls its templating Super and it works just like this — directives are written in HTML rather than on top of it.

You would likely use other attributes aside from id, and there are a large number of JS based templating frameworks that do this.

https://github.com/Floofies/cdaTemplate

The Zope application server uses something like that, ZPT (Zope Page Templates) and TAL (Tenplate Access Language), IIRC. Plone, a CMS built on Zope uses them extensively.

> Any thoughts?

10–50 pages per second is already more output than a human can produce input for, so it shouldn't be considered a problem.

enaml-web does this using lxml. It works fine for a limited number of concurrent users.

Probably raw Svelte can be used as a frontend template to your backend

PHP.

I started my professional career writing PHP and the deeper I go in everything else, the more I appreciate the fundamentals that PHP defines as it’s happy path.

php (itself!, not using a library) would be string based instead of DOM based, so the opposite of what OP is asking for.

The end of this road is React's Server Side Rendering and Next.js.

This approach only really has one major benefit over string-based templating, namely, it lets you re-use code across server and client side DOM logic.

Type-checked template application is a benefit.

Reply on news.ycombinator.com