Htmy – Async, pure-Python rendering engine | Hacker News Reader

Htmy – Async, pure-Python rendering engine

volfpeter.github.io

146 pointsfriendly_deer87 comments

Not clear why HTML rendering needed to be infected with async. None of the example code has a clear need for async - even the `is_admin()` method would be a prefetched property in any reasonable database model.

Your counterpoint still naturally involves something like async _somewhere_ (your proposal is just to move it out of the HTML rendering and into an initial data-gathering stage). If you accept that premise then the question is just where the async code goes.

While on some level it makes sense for HTML rendering to be a pure function where the inputs are gathered from elsewhere (potentially asynchronously), it looks like htmy wants to make it easy to define hierarchies of components. Instead of `is_admin()`, imagine a dashboard whose layout is stored in a database, supporting configurable charts of various flavors. The heterogeneity of the data supporting different types of charts makes it hard to efficiently pull data in a single SQL query (equivalently, any reasonable database model), so somewhere in your code you're pulling a bunch of data asynchronously, and somewhere else you're rendering it. The question, still, is "where?"

Going back to the idea of htmy defining hierarchies of components, imagine how annoying it would be to have to manually grab all the data for a "reporting page" component only to feed it straight back into the renderer -- either having to duplicate the hierarchial structure when feeding data into the renderer (a technique some UI libraries employ, though I don't like it) -- or having to come up with a method for flattening that hierarchy when instantiating the component (another technique some UI libraries employ, one I like more for small projects and less for large ones).

They solve that (to the extent that you think it needs solving) by bundling all that background logic into the components themselves. Did they really need to implement that recursively instead of just walking the hierarchy, gathering the data up-front, and populating it? Eh. The code winds up being similar either way, and either way it definitely forces async back into the middle of HTML rendering.

Mind you, that tends to either make some applications hard to build or to cause the framework to explode in complexity over time as people need new and new ways to say "yes, re-render this thing; no, re-render that other thing, but don't grab its data, ...." There's enough less particularly annoying code involved though that fat, smart components are a natural place for people to gravitate.

Unrelated to htmy completely, a technique I like from time to time even for problems which don't need async per se (and I'm usually using lower-level languages, so the implementation is some sort of more manual continuation pattern, but all those things are basically async, so I won't dwell on the details) is explicitly designing pausable/restartable structures for long-running computations. It's about as easy to write as purely iterative code, and you can run the result as purely iterative with no runtime overhead, so the downsides are low. It opens the door though to easily tuning how long you defer invariant maintenance (too infrequent and your algorithm devolves to the slow thing it's replacing, too frequent and the overhead isn't worth it), easily checkpointing a computation, adding other custom runners for an algorithm (like animating its progress), .... I can absolutely see a use-case for wanting to visualize each step of an HTML rendering, or log OS network counters after each step, and so on. Python's async isn't really the right tool for the job for that (it's hard to modify the runtime to support them without building quite a lot of nonsense from scratch), but async in the abstract isn't bad at all per se.

Imagine you have 2 big components, one fetches from an third-party API and the other from your backend, this way they can load at the same time instead of sequentially.

Because checking for two conditions is impossible? This seems like a solution for a non-existent problem. I could be missing something

I was imagining more like you have a Django view that does all the async data fetching and then you hand off the results to a 'dumb' page component that does only rendering

I guess the point is to have components know how to fetch their own data, particularly when combining with HTMX and having backend return page fragments that correspond to components. But maybe this makes more sense in React than it does when translating the pattern back to server-side?

e.g. same author has this https://github.com/volfpeter/fasthx?tab=readme-ov-file#htmy-... which is doing that, but there's still a 'view' endpoint. Why not put the data fetch code there and have 'dumb' components that don't need to be async?

It seems like the view endpoint would be for functionality shared the full view, like auth safeguards and such, while the components would fetch the data they need; this would make it so you don't need to pass around the data to the view and save a few lines of code; of course this is not compatible with the idea of having "dumb" components vs "logic" ones like people do in React and alike.

Components don't really need to fetch anything, they don't need to be smart. It's up to you where data fetching happens. If you look at fasthx for example, you'll see that routes/views normally handle your business logic and fasthx does the rendering (now with Jinja or htmy). With Jinja for example, it can only work like this. With htmy, you have more flexibility (which can be an advantage but of course it can also be misused).

Async components can be handy for example when you need to load files. See the Snippet utility and the markdown support in the lib. https://volfpeter.github.io/htmy/examples/markdown/

You're right, fetching all the data (that you may or may not need during rendering) in advance is of course doable and quite common. That's what you do for example with tools like Jinja. That may or may not work well for your use-case.

htmy does not force you to put data fetching or anything else into components (you can still have dumb components). It also doesn't force you to write async components. The most important thing it does is it gives you the option to build your application entirely in Python (no ugly custom templating language syntax with lack of static analysis or proper IDE support) and enables the use of modern async tools.

And admittedly, the lib was built with FastAPI and HTMX in mind, which kind of necessitates async support...

My comment was just thinking out loud really...

It seems like if you're not doing data fetching in the component then there's no need for it to be async.

And then I was wondering if maybe data fetching in components was a good pattern. It's quite different from what I'm used to doing.

Yeah, the renderer itself must be async to enable async tooling, but everything else can remain sync unless async is really needed.

Regarding data fetching (it's a recurring theme in the comments), I'd probably do most of my async business logic in my routes (well, I'm obviously using htmy with FastAPI and HTMX through FastHX), and components may fetch additional resources if they need something other than the route's result (translations, markdown, html snippets, some other IO).

I'm not sure if any other tool really enables this pattern, but I'm quite curious to see how I'll use it in future projects, and hopefully also what ideas and patterns others come up with in their projects. There's definitely room for creativity.

But does it actually work that way? If I `await fetch_from_api()` in the first component before returning the tree with the second component that fetches from my backend, `fetch_from_api()` has to resolve before Htmy finds out about the second component.

You’d have to structure it differently, it’s definitely not a free lunch

Right, but if this framework enabled that optimization, I’d expect it to have some examples showing how you’d implement it. As far as I can tell, there’s no way to structure your code to allow parallel component rendering with this framework.

With the current renderer (which is super basic because simplicity and features were the main priority over optimization for now), if a component has multiple async children, they will be resolved concurrently. I assume that's what you meant by "parallel". Not sure why that would need an example, but you can simply create a component that calls asyncio.sleep(1), then create a bunch of instances and render them to test it.

I want to have at least the same level of concurrency when the new, better optimized renderer lands.

Async infrastructure allows your stuff to be sync or async. While sync infrastructure forces your stuff to be sync.

If anything sync (not async) infects everything you do.

Of course it depends if you call the infrastructure (then it's better for it to be sync) of if the infrastructure calls you (then it's better to be async).

Rendering engine is something you rarely call, but it often calls your functions.

Yes, and that’s the worst part of async. That’s why you need to be very strategic about where you introduce it into your code in order to minimize the number of functions it infects, not give up and write a framework that’s all async for no good reason.

https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...

Yes. But you should be equally strategic about introducing sync code into your platform. Because making your platform sync basically makes it only be able to call sync functions of your code.

It's not that async infects. It's sync that infects and restricts. We are just used to it by default.

The fact that we started from sync was the cause of all the trouble of rpc because everything outside of CPU is innately async.

So make your utility functions sync whenever you can but make your platforms and frameworks async.

I just completely disagree. Async is syntactic sugar that can be reduced to sync code with callbacks. It doesn’t exist on equal footing. If you want to call sync code from async code, you just… call it. If it performs blocking IO, it’ll block, but that’s exactly what it would do if called from other sync code, too.

By contrast, calling async code from sync code requires a special blocking wrapper (Python) or unavoidably breaks control flow (JavaScript).

> By contrast, calling async code from sync code requires a special blocking wrapper (Python) ...

That's exaclty my point. If you don't have async by default in your platform you need to do stupid things to fake it. If function calls and main in Python were innately async you could be calling async code just as easily as sync code.

> [...] or unavoidably breaks control flow (JavaScript).

async/await syntax avoids it completely.

Tbh await should be default function call semantics and there should be special keyword for calling without awaiting. But since we come from sync primitives that would require small revolution that might happen at some point.

> Async is syntactic sugar

You could make sync code be syntactic sugar for await.

> would require small revolution that might happen at some point.

or python could have blessed gevent and done away with all the nonsense.

I hope that someone does an oral history of why gevent wasn't seen as the solution here. The existence of models like Twisted, and a general idea that yields to an event thread should be explicit in some way, I think caused the exact kind of fracturing of the ecosystem that everyone was trying to avoid. "Everyone will write async code" simply didn't happen in practice.

So many production problems that we never seen in development comes in and no way to debug

You had never tried gevent in production then. As soon as workload and concurrency increases python programs with gevent or gevent based drivers, especially monkey patches causes unexpected crashes out of the blue, no way to debug, no error message,Memory leaks and whole slew of nightmares

> Tbh await should be default function call semantics and there should be special keyword for calling without awaiting.

Your comment made me realize this is exactly what golang "go" keyword does. This is actually great.

also Cilk spawn.

> Async is syntactic sugar that can be reduced to sync code with callbacks

The whole point of introducing async was to get away from callback hell.

Yeah, I’m not saying that callbacks are good, I’m saying that async is a veneer over callbacks.

> Async infrastructure allows your stuff to be sync or async. While sync infrastructure forces your stuff to be sync.

Is that specific to the threading model for Python?

The reverse is true in nodejs where once you’ve got one async call, the entire chain must be async.

Python is the same as JS.

Async function (that returns something you need) can be called only from async function. That's why autor of this specific rendering framework/lib chose it to be async. So that the user functions called in components can be either sync or async.

Thanks for this answer. Async support is handy if the framework in which you're using the tools is async (let's say FastAPI). See my answer to a similar question on reddit: https://www.reddit.com/r/Python/comments/1fvv11p/comment/lqb...

There's a bunch of these kinds of html renderers. Here's mine: https://pypi.org/project/simple-html/

But there are many others. Not sure I understand the point of async rendering, unless you want to mix IO with rendering? Which feels a bit too close to old PHP+html for my blood.

What's wrong with the old PHP+html ways? It's one of the best toolchains to knock out a small to medium sized project. I guess that fundamentally, it's not scalable at all, or can get messy wrt closing tags and indenting. But with this approach I think you're good on both these aspects?

For websites you make for Tor, you would typically go for PHP or OpenResty, as it needs to be JavaScript-free. I personally aim for JavaScript-free projects regardless.

Of course if you want client-side whatever, you need JavaScript.

JavaScript is optional even on the client side nowadays with the advent of PyScript via WASM, etc.

I did not know that. Is it true? Can I have dynamic updates (something like what AJAX does) without refreshes? If so, I need to do some research in this area! I assume I can use any programming languages for WASM as well?

Sure, dynamic updates are possible. Re language support, I'm only aware of PyScript for Python, and Blazor for C# already being fairly mature. But there are other language ports in progress.

What do they call this these days, dynamic updates using WASM?

I can't clearly see a use case. I went on to the "why" section but I'm having a hard time trying to understand what this is trying to solve. Perhaps a clear and simple example to see why you would use it could be useful. Also I find it extremely verbose to write HTML the way is shown in the examples at the top. Having used Jinja for a very long time, its simplicity and separation from logic makes it almost (for me) the only templating lang that you need to learn in Python. Writing HTML code the way is shown is clearly not for me, but there might be uses cases for it.

Sometimes I want to do things totally pure html, with more dynamicness and more reusability. Jinja template fall short.

This! I recently wanted to get back to writing a web app entirely rendered on the server side without the need for a JavaScript framework and I was really struck by how embarrassingly clumsy templating engines are compared to JSX.

That's a fair point, although my feeling after working quite a bit with Jinja recently is the opposite (primarily for lack of static analysis and IDE support).

You're right, for example the documentation should be improved quite a bit. Keep in mind that this project is pretty new, I simply had no time to add more docs or further improve the existing one.

Ps.: with the Snippet utility and markdown support, you can actually write quite a bit of your HTML in a html files rather than Python. You could even use Jinja templates as the backend for some of your components. This part will see more work as I have spare time to work more on the project.

There is already htpy: https://htpy.dev/ I have used it in production and like it.

For those asking the point is being able to do similar to React JSX components, but on the server side. It's so much nicer to use than templates like Django or Jinja (there might be other reasons, but this is quite clearly the goal of htpy and I assume this too).

Just looking at this one briefly it seems to use magic methods on dataclasses. What's the advantage of that over just a function? Seems like unnecessary nesting.

I've seen htpy before starting this project. While creative, I'm not too happy with the interface if I'm honest and it feels quite a bit more limited.

There are no magic methods really, you can even write function components. Using dataclasses in examples is also an irrelevant technical details.

The actual reason for requiring an `htmy()` method is that this way you can turn any of your business objects (be it Pydantic or SQLModel classes for example) into components without the fear of a method name conflict with your business stuff. Actually, I expect/planned this to be a very frequent use-case, and then there'll be zero unnecessary nesting.

Ah OK, that makes sense. I hadn't really thought there was much room to do things differently than what htpy does, but I probably just haven't thought about it enough. I'll definitely give this a go as well. I think the idea in general is a good one.

Oh it's server side "rendering"?

To me “rendering engine” also means something else. Namely taking html and rendering it to the screen.

Consider updating your vocabulary because the term is often used for both.

Note that rendering to the “screen” really means writing bits at a memory range, which is just one interface for displaying things. Html is another, higher level interface these days.

"rendering engine" has a pretty clear meaning and is a pretty poor term to use for a system for tranforming one kind of text bits into another.

Perhaps you should consider using less confusing terminology in your vocabulary?

https://developer.mozilla.org/en-US/docs/Glossary/Engine/Ren...

Edit: You say "often used for both" but I am struggling to find any other examples. Yhe closest I can find is this extremely poorly named static site generator project: https://github.com/render-engine/render-engine?tab=readme-ov...

Edit2: Man, the appropriation of the term "rendering" by JS people has led to some pretty stupid stuff, like this statement: "SSR, short for Server-Side Rendering, is a technique in web development where the webpage's content is rendered on the server instead of the client's browser."

https://en.wikipedia.org/wiki/Server-side_scripting#Server-s...

This is what I am looking for.When FastHTML was announced I expected to work like this one but it was with it's own webserver.

Funny, I went through the exact same process before I started creating this project :)

Hypertext markup Yanguage?

But what do you do use to create dynamic updates on the client side? I’m guessing it still has JS and makes API calls, no? And if so, it seems easier (to me) to just do all of the rendering client side and let the backend just be REST queries.

This is just the static html renderer, it has no JavaScript to update client side, but the author has another project for fastapi + this + htmx: https://github.com/volfpeter/fasthx

I was looking for something like this a few weeks ago. I typically use Django and hate the template engines limitations. I needed to make some reusable components and the best option available was switching to jinja to get their macro support, bleh.

This reminds me of the best part of Flutter UI composition, but in a language I always return to.

Have you done any benchmarking? I don't even know what the comparison would be.

You might want to look at django-cotton for components

Check this out: https://compone.kissgyorgy.me/

Much simpler than this library, components are simply functions, rendered to strings.

I made one microbenchmark, it's "only" 2x slower than Jinja2 right now, but I know how to make it faster.

if you can make it as fast as jinja2 I'm sold ...I haven't done my own benchmarking but so far I haven't seen any of these HTML-in-Python libs able to report comparable performance

I've implemented a bunch of AlpineJS "components" as jinja macros in my current project and ... it works, but it's pretty ugly and it sucks not having type safety or ability for the IDE to understand connections between the template and the Python code

what I really want is something like JSX/TSX for Python... having gone through this process I can see why that approach is desirable. I kind of feel like libs which mimic the syntax but unable to provide the type-safety/IDE support are missing the point. So although I love the look of "Python HTML element objects" approach libs like yours and OP have I think for now it is probably the best way available.

for my current project we are pre-compiling all the jinja templates (via Jinja's own utils) for deployment as AWS Lambda

I did look into JinjaX but it has its own separate jinja env and secondary template cache and didn't look like it would be easy to plug it into the pre-compile step

+1 for TSX for Python, that would be great!

How many of these are there? I also pointed out htpy elsewhere in the thread.

I haven't done benchmarking yet. To be fair, I had limited time and I focused on developer comfort and the features I needed for projects I work on. Simplicity and flexibility was another goal: the rendering engine itself is as minimal as possible, but can be replaced or optimized in the future.

I'll probably do a simple comparison with Jinja (using FastAPI) this week. Given that I can put an htmy() method on my business objects (it was an important design consideration, no conflict with other tools), I expect an okay results, but we'll see.

I did some testing in the meantime. Depending on what you render, it's about an order of magnitude slower currently.

The renderer is as simple/minimal as possible at this point (the focus was on the core feature set I needed until now), so it's performance is as bad as it can be :) There's plenty of room for improvement. I'll work on a few optimizations as I have time, but contributions are more than welcome.

How does this compare with FastHTML?

Slower because there is no fast in the name.

Misleading and against hn attitude

The most important difference is that htmy does not bring its own web framework, you can use it your preferred one (preferably one with async support, but you can always delegate the rendering if you use a sync one).

Would love to see some benchmarks for all these libraries that compare them to Jinja2.

They are almost always slower, because jinja uses some compilation tricks. But, it usually doesn't matter compared to remote database access.

In my experience it doesn't matter if templates are pretty simple, or the database access is not optimized. Can matter if you're not careful.

I just noticed on Reddit that someone posted my package here. I see there are several comments already. I'll try to answer a few as I have time.

I like the htmy method that an object can render itself, neat idea. But the extra classes for rows etc seems too bureaucratic.

Looks great. Anyone using this in production?

Is there a comparison or guide to choosing python frameworks? Every few weeks there's a new one posted here

In the real world, for web things, people use django or fastapi. I'd suggest picking a project with lots of stackoverflow questions and poking around their docs to see which makes you the most comfortable. Personally I tend to favor litestar these days since it has good docs and issues don't sit around for years waiting on one dude to merge prs (fastapi) and it's a lot nicer than django (and I hate django docs).

Flask/quart are painful to work with due to horrible documentation in my experience, but they're popular too. Quart is just an async rewrite of flask by the same owners.

Litestar has a half baked comparison chart here: https://docs.litestar.dev/latest/

I think the “rule of thumb” is that none of them are better than using HTMX with templates. HTMX obviously having some limits in terms of security and complex REBAC.

Or Unpoly. I've been working with it for a month now and it's a real pity such a robust library it gets so little attention.

HTMX + templates are complementary to a backend framework rather than an alternative to one

Not a comparison, but a fairly comprehensive list that I maintain, with github stars as a proxy for popularity:

https://github.com/sfermigier/awesome-python-web-frameworks

Note: as you probably know, popularity is not necessarily correlated with "actively maintained". For instance, Hug and Sanic are quite popular, but haven't seen a commit for quite a long time.

Rendering html is something that needs to happen within 300ms. Anything more and its perceived as lagging. So why would you choose python to do visual rendering?

It could be useful for content creators that value reusing their Python expertise over other factors. Also, many apps work better when every integrated component is written in the same language. Also, there’s a lot of code in Python for or supporting web programming. Finally, if people use AI auto-complete, many people say they’re more effective at common uses of Python vs other languages or situations.

I’ve found the performance issue to be serious in some situations. Fortunately, there’s a number of accelerators for Python code that boost its performance. They range from JIT’s (eg PyPy) to custom VM’s (eg Cinder) to writing fast paths in Cython to Python-to-C++ compilers (eg mycpp).

So, you get the productivity and familiarity of Python with performance boosting in many use cases. If it doesn’t work, then it’s better to write that component as an extension in a systems language.

Looks similar to a framework I've been using for some personal sites reflex.dev, pretty cool when would you recommend using this over that?

That's a pretty complex question.

Reflex is a great project with a great feature set, it does everything (client rendering, state sync, API) and you can even write your callbacks in Python. It seems like the best option from this family of frameworks (alternative is NiceGUI for example, but having worked quite a bit with that, I probably wouldn't recommend it). Doing everything has some downsides though: there's a ton of "magic" under the hood, the lib is obviously very opinionated (it couldn't exist otherwise) and you may have a hard time if you need something that's not built in to the framework.

htmy is pretty much the opposite, it only does HTML rendering and comes with a set of utilities for advanced uses, e.g. async support, context usage, styled markdown, etc.. With FastHX, you also get a pretty convenient, declarative integration with FastAPI and HTMX. The tool is ergonomic, but you do need to put in more work compared to Reflex (create APIs, use HTMX, maybe AlpineJS or similar client-side tools). In exchange for simplicity (and lack of magic), you get full control over everything: you can convert your business objects to components, use any CSS/UI lib, any backend tooling. Extra benefit is you can migrate to (and from) it relatively easily from tools like Jinja.

I think in almost-2025 any dataclass heavy library should probably use pydantic (or support it)

Probably, but I fail to see how that's relevant here. This is not a "dataclass heavy" library in any sense, they just used dataclass in the examples to make them shorter.

Based on everything I see in the documentation, you should be able to use Pydantic models as well, or standard python objects, or anything else, as long as it has a method `def htmy(self, context: Context) -> Component`.

Please don't! Pydantic demands 100% type correctness at runtime in a language that can't guarantee basically anything at "compile" (lint) time. Screw up one type annotation for one edge case and your entire system turns into one big ValidationError.

Dataclasses let you return "incorrect" data and that's a good thing. I'd rather get an unexpected None here and there (which can be handled) than have library code crash because the wrong type snuck into a field I don't even care about.

As for support, is any explicit support needed? You can Pydantic models into things expecting dataclasses and often the other way around too.

Spoken like a true dynamic types programmer. Some programmers prefer having errors over these surprises.

Reply on news.ycombinator.com