Show HN: Generate styled web pages with just Python | Hacker News Reader

Show HN: Generate styled web pages with just Python

pyvibe.com

229 pointszainhoda84 comments

There are a lot of Python to web app frameworks going around these days but I wanted something that was a little more lightweight that just generates HTML pages and can be embedded in Flask or other Python web servers incrementally.

PyVibe uses Python components to construct a page with styling that you can use in Flask, in a static site, or even in Pyodide.

Open articleView on HN

I noticed:

in the source of the homepage. Just FYI the tailwind people say that's just for dev mode and never deploy that to prod. Instead you gotta run something like:

npx tailwindcss -o ../assets/css/tail.min.css --minify

and then replace the script with:

"a script referred to as the Tailwind Play CDN. This script is only to be used for learning and experimenting, never for a live site."

from https://www.codeinwp.com/blog/tailwind-css-tutorial/

Yeah, the reason for that recommendation is mostly to reduce file size. For now I don't want to have an npm dependency since this is geared towards Python developers. I'll eventually build in the tree shaking directly into the Python library.

I have the same problem with my project! go vs. python but yeah I hated adding the npm stuff. Let's port tailwind npx command. I'll work on the go version.

I haven't tried it yet but I was planning on using this library to do most of the heavy lifting: https://pypi.org/project/treeshake/

It looks like it's calling C code via Cython so if it works, you could probably use the same underlying C code via CGo for your project.

As someone who currently runs Docker inside an M1 Mac: I agree with this: all commands should be written in Go. So much easier to install everywhere.

I have a somewhat neglected project to package Node.js for PyPI. Perfect for you type of project, the idea being you can have it as an optional dependency that gets installed into the users venv, and you can the run tailwind from Python.

https://pypi.org/project/nodejs-bin/

Very interesting! I’ll check it out — it looks quite promising as a solution

Note that tailwind now has a standalone binary: https://tailwindcss.com/blog/standalone-cli

Ah that’s great to know thank you.

Third-party CDN usage like this isn’t a best parctice generally if it can be avoided because it involves users need to trust more parties and leaks user data to those sources. With the advent of assets being isolated a couple of years ago, there's no performance/chaching benefit to it anymore either.

Also: if the CDN for some reason is non-functional it impacts the reliability of your service. That might be a short outage or a complete shutdown of the server.

Unless you plan to monitor that stuff constantly and unless you tested your program with single components (or all possible permutations) missing, hosting these yourself might be the better option anyways.

Additionally your users don't signal their visit of your page to a hundred other servers which certainly isn't the most trustworthy behaviour.

I hear you folks. I’ll add an option for local assets.

That should be the default for security/privacy. It's better to have a CSP with just first-party trust too.

I love new technology, and this is really cool. I don't want to derail the discussion too much, but I've noticed this trend of "making web dev easier with just Python" - e.g. htmx, pynecone, streamlit, etc. The common theme seems to be magically hiding JS from the developer.

Can any python devs who use these tools explain the appeal? Do you just have an aversion to JS? And how do you reconcile that with the fact that, at the end of the day, your code is running in the browser as JS, CSS, and HTML? As soon as you need to debug something, you'll be a fish out of water. To me that sounds like a "worst of both worlds" kind of situation.

Is JS really that bad?

I use htmx, and I have a good amount of experience in the JS world. The main issue with JS frameworks like React is that it's painful to integrate into things like Django-rendered templates. It can definitely be done, but the tooling just isn't there yet, although it's way ahead of where it was even 5 years ago.

You can go the SPA route and use something like Django Rest Framework with React, but then you need to build API endpoints for basic plumbing like authentication that often doesn't warrant an API. And you lose the use of a lot of the batteries-included features of frameworks like Django.

If there were an ergonomic way to integrate React components into specific areas of the DOM tree rendered by Django templates and a sensible way to pass information into props from Django, I'd love to use a mature framework like React over things like htmx.

I've been away from direct work with Python + Django (P/D) for a couple of years. One of my issues with P/D has always been that deployment on anything other than one of the specialized services is an absolute pain in the ass.

In the PHP world, you can get a simplest of servers from any host, upload the files and you are up and running. With P/D, well, it's a lot of work.

I have developed this opinion that, while useful, Django's SQLite and runserver approach for getting up and running quickly actually is detrimental. To run a real site you have to break all of that, deal with static content, etc., etc. I could also argue that the default directory structure, while simple, it is also detrimental when moving to production.

In other words, sometimes I feel it would be far better if P/D initial setup was closer to a canonical installation that could be run on any server with something as close as possible to a simple FTP file transfer (just like PHP).

One can always make things more complex (multiple servers, load balancers, etc.). However, I truly believe the P/D ecosystem is seriously missing the point when not delivering a 5-minute install that actually is a basic production-ready install on any server.

Have things changed at all in this regard over the last couple of years?

That's true, and PHP certainly has ease of deployment over Django. I generally deploy Django applications on services like Heroku or Render, but it can get expensive since you can't really re-use instances for multiple applications like you could with a VPS and multiple PHP applications.

But, at least to me, ease of deployment is less of a concern once it's going. And since I stick to one hosting provider and only have a handful of applications I maintain, its not something that crosses my mind until I spin up a new Django application which happens only rarely.

I often joke that Docker wouldn't exist if it had been easier to deploy Python code, since it was DotCloud who invented Docker as part of solving that problem.

I was hoping Vercel would be their spiritual successor, back when they were known as Zeit and allowed running arbitrary containers at the edge, but alas they pivoted to focus only on the JS platform. That said, IMO Next.js is the closest thing to modern day PHP in terms of filesystem-based routing and "write code on a page, render it at the same URL."

For Python, we run our services in containers using some custom images, and build the code from a monorepo using a combination of Pants and Poetry. For smaller or more focused projects, I think Google Cloud Run is quite nice, since it supports the Buildpack Spec and is in many ways a successor to Google AppEngine.

Something many people miss is that JSX and React are two separate technologies. JSX is a fantastic templating language in its own right.

Have there been any libraries that attempt to integrate JSX more natively into Django templates? It seems like that might be an approach that could solve the ergonomics problem you describe.

These are just frameworks that allows you to make an easy interface for your mostly-Python-based code. The problem is never that “I only want Python”, but “I want a web app but I don’t know much JS”.

It's not so much about having another language to learn, it is about sharing code.

Suppose you need a meaningful data structure to encode something very domain-specific (for example a dynamic filter you could build with arbitrary boolean operations). On the front-end side, it will be like a user-facing DSL that you can build with UI elements. On the backend a recursive algebraic data structure you need to parse and that should go through standard algorithms to obtain (most likely deterministic) results.

So what happens if you have two stacks? You'll need to code a number of these parsing and processing algorithms twice, and each inconsistency will be a bug. Every time we had the need for something like that in a project, it was done in the backend easily and correctly, but the frontend was lagging and missing features. So even if the backend language is transpiled to run in a browser, it does change the game to be able to run the correct algorithms directly.

My co-founder and I originally divided our backend/frontend work between each other, and between Python/TypeScript. We found the sweet spot for solving this code sharing problem was GraphQL with code generation on both sides (resolvers from types on the Python side, and types from schema on the TypeScipt side). Most backend services are in Python, using Strawberry for strongly-typed GraphQL resolvers, and client code is in Next.js using TypeScript, Apollo GraphQL, and graphql-codegen for autogenerated types from the GraphQL schema. (We also use Postgraphile as a sort of naive base layer for basic operations, and later added schema stitching to have one GraphQL API in front of multiple services, but that's a longer story [0]).

In our company we generally divide the work between frontend and backend developers, and we've found that we usually complete the backend implementation for most features before the frontend anyway, so the GraphQL schema itself becomes part of the spec. But more importantly, GraphQL becomes a unified type layer shared between languages. Backend and frontend developers can each work with the most native features of their respective languages, rather than some subset of each that's supported by the other.

The code generation is the secret ingredient, because it lets us draw a straight line of automation from Python types to TypeScript types, with no need to manually write any middleware. Usually frontend and backend is implemented by a different person, but in the cases where one person is doing both, they can literally save a Python file and wait for a chain of hot reloading to have those types available in the Next.js app.

It's a pretty awesome setup and I'd recommend looking into building something like it if you're working on a greenfield project and intend to grow your team of developers. But it takes a lot of effort to setup. We arrived here incrementally, originally starting with just Postgraphile and later adding the Strawberry resolvers and schema stitching. If you're an individual, it's almost certainly an unjustifiable amount of work to setup. And it might even be too complicated for an org of multiple people, if they typically have the same full stack developer implement the front and backend of a feature. But for us it's been a great experience overall. And in general, I absolutely love the combination of GraphQL and type-driven code generation, and I would recommend that even individual developers find a way to leverage that aspect of it.

[0] https://www.splitgraph.com/blog/graphql-schema-stitching

In one case we had a setup close to that.

But the case I'm talking about is when you'd have something too complex to encode simply in a standard type system (even with dependent types...). So you have some subtle relationships between different fields, or some specific subtree might be a forbidden substructure, etc.

Reimplementing algorithms might be necessary on the frontend for interactive purposes.

For most standard use-cases I agree you can have this well-define separations between backend and frontend.

Yeah, I think I know what you mean, and we have a case of this in our search system where Facets are dynamically defined, and available facets are sent in each response so that subsequent requests can use them in their payload. Ironically enough that is the one backend service that is written in TypeScript, and does share some code with its frontend implementation for just the reason you describe.

We also have plugins for importing data where the parameters are dynamically defined with JSONSchema, and we had to do a whole bunch of work to make validation and form rendering work properly on the frontend, while also duplicating it in Python, but luckily we can mostly fall back on standard JSONSchema libraries for that (we also use superstruct on the frontend). We have to rely on getting it "mostly right" in the frontend and then trusting the backend to send back an error message on failed validation (and then we have the problem of rendering that error message in the right place).

Generally speaking this still bites us with any GraphQL resolver that has a field returning JSON for whatever reason. The major aspect of code sharing we're still missing is validation of that data between front and back end, and I can see how keeping all your logic in Python would help with this if you're both rendering and validating forms from the same Python code. Of course you can get the same benefit from writing your backend code in JS and sharing that, and I guess that's one of the downsides of our approach, since we lose out on both options.

Is it that bad? No. But that doesn't mean I want to do it.

I'm a more casual python dev. I make small tools, solve unique/corner issues. Nothing fancy. It makes my life vastly easier for many of these tools to be webbased, typically because I want to hand them to someone else to use so I don't need to think/worry about it.

Context switching between languages is very very expensive for me. I used to write a lot of nodejs and Python but I'd find myself either forgetting or adding semi-colons, confusing which for-loop style for which language. Eventually I came to the conclusion it wasn't worth the mental gymnastics to switch and gave up node (aka Javscript). So can I write it? Yes, but I don't want to.

These new entirely python based web apps are really interesting to me. They don't have to be super advanced because I don't need that. It greatly simplifies the process and the time required.

Also, almost no one writes "just javascript" any more. It's htmlx, react, yada yada.... and they've all got some cool new build system. Seems like every time I look, there is a new build system people are using. Which I don't want to learn or deal with.

This describes me to a T. I think if GP saw the js I write, they would understand why I don’t want to write it. But it works, I can get fairly straightforward stuff done quickly, and I’m the only jerk who has to maintain it.

>Is JS really that bad?

No, but npm is. If you shelve your vue/react/etc project for six months, then go to build it on a new machine, you'll find all kinds of things are broken. Contrast with python and one or two smaller js libraries, which will not require any updates or fixes for much longer.

> If you shelve your vue/react/etc project for six months, then go to build it on a new machine, you'll find all kinds of things are broken.

This is not true in my experience (I literally did exactly what you describe recently). I use Yarn, with pinned dependencies and yarn.lock checked into the repo. Everything works exactly as expected after 6 months, because the hashes of all the packages match each other. You just need to make sure to run `yarn install --immutable` when you come back after 6 months, to make sure you don't update the lockfile (and pinning dependencies to their exact versions helps avoid installing a different version within the same semver range). Now, whether everything will "just work" after upgrading one of the dependencies is a different story. But if you don't touch package.json or yarn.lock, then everything should work fine.

> Contrast with python

I'm not sure Python's dependency management is a great hill to die on. But Poetry is pretty good. In our project's monorepo we've settled on a combination of Pants and Poetry. But it's far from easy to setup, and I'm not sure the default behavior has the six months of stability you describe, in comparison to Yarn (but I'm not the Python guy, so I don't know the details. I've just noticed a few occasions where installing packages unexpectedly modified the lockfile; take that with a grain of salt, though.)

What is your Python package manager setup (just curious)?

>What is your Python package manager setup

For this project, literally just pip with a requirements.txt. Poetry is great though.

As someone who knows Python and always has to google CSS properties and javascript functions, this is a really sweet little project! I'm not going to reach for this if I need to have a website that needs to look a very specific way, but for cranking something out to display some data or documentation, this seems really nice.

Exactly! The use cases I've been hearing about are:

- You're a data analyst and you did some work in a Jupyter Notebook and now you need to give it a front-end so that other people at your company can view/interact with what you've built

- There's a Python library that's runnable as a command-line tool or importable but you want to give it a front-end

I work with data and keep having the first use case, and it's pretty common in my company. So far we're using Streamlit and Rshiny (bilingual data org). We're not really happy with Streamlit, since it's pretty easy to grow out of its pretty single minded way of working.

I'll definitely be checking this out!

what are your pain points w streamlit?

Interesting. I’ll try it out with my students. Doing HTML and CSS in a Flask app seems advanced to them. I can recommend a “deploy to PythonAnywhere” guide so users can get it online. That would require your module to be part of PythonAnywhere’s environment.

What’s your timeline? As it happens, we’re also working on a Python hosting solution with a generous free tier.

Do you have any additional information to share about the hosting service? Will this be a static host, a Python-first PaaS, an education-focused REPL environment, etc?

Hey there! So it’ll be a Python-first PaaS focused on running data apps. So it’ll include storage for pickle files, a simple NoSQL database, identity aware proxy so you can configure specific routes as being authenticated, secrets manager, and the ability to run background jobs on a schedule (usually some sort of ETL process or model training).

I go back and forth on the REPL. We built a REPL but then a lot of the beta users preferred the power of GitHub Codespaces so right now I’m thinking that no, it’ll not contain the REPL.

If you have a use case, would you like to beta test the hosting?

Oh, that could be fun. I’ll give it a spin.

I’m working on a something like a structured-data first wiki. Most users will be unauthenticated read-only, served well by a static site. A smaller population will submit new data and edits. Submitted data will need some processing pipelines.

I’m using Dolt/Dolthub to publish high fidelity data dumps. I’ve been leaning towards doing a static site built directly from Dolthub queries, then use an offline workflow for modifying data. However if I can bring those together easily into something more like a standard dynamic site, that would be useful.

For example, I could use your native data storage for basic user profile and authentication information, like API keys for Dolthub.

i feel PythonAnywhere is a great concept but less great execution.

This reminds me of https://pynecone.io

I would love to see these merge somehow if possible.

kinda but this doesn't have a web server built in. nor does it depend on react or anything. so you can just use flask or whatever else.

Mmm, looks like the library doesn't do any escaping for content (and doesn't offer a way to do that either)?

what do you mean by "escaping for content"?

I think what they mean is that methods like add_header and add_text actually take raw HTML as an argument, which means that if you want to add arbitrary text, you actually need to do

    import html

    ...

    card.add_text(html.escape(my_text))

That’s correct. Perhaps I should add some documentation around this. Untrusted user input needs to be sanitized before using it in the page.

Seems like that should just be a warning in the docs, since mostly we’re talking about trusted code written by the dev…. Although maybe that’s untrusted from your perspective as a PaaS?

What’s the escape hatch to write custom HTML in the Python source?

there's "HTML (.add_html)" in the docs - "Renders raw HTML. This is meant to be an escape hatch for when you need to render something that isn't supported by PyVibe." I'm guessing this isn't sanitized and we trust the dev. After all, they can do "eval" in Python which is much riskier than raw HTML :)

All of the components support rendering arbitrary HTML, which is exactly the problem.

No escape hatch is required since nothing is escaped...

So cool! Is there support for layouts?

Edit: Aw broken link - https://www.pycob.com/components

Edit: Nevermind, found it - https://www.pyvibe.com/component_reference.html

Sorry about that! Where is the broken link? I can't seem to find it

what do you mean by "layouts"?

The HTML markup in the form example is not valid. Label for attributes correspond to the input's id – not its name.

I'd be really hesitant using a technology that abstracts away something as fundamental as HTML. You're putting a lot of faith and trust into a library that will make mistakes that you can't fix.

I'm curious about the performance implications of using a Python-centric approach for creating web pages. Would the generated HTML pages be as efficient and optimized as those created with traditional web development techniques?

What is "traditional"? In Django the web pages are generated by Python, too. In Rails by Ruby, etc.

I wonder what it would look like if websocket support was added. Perhaps add_* would return Sink objects which would have appropriate update methods that could be called in another thread? But that would require a page.run() function

I'll give this some thought and I'll DM you on Mastodon if I can come up with a working example

Here's a quick example of how Websocket components might work:

https://websocket.pycob.app/

Let me know what you think! We can discuss more on our Discord: https://discord.gg/uQWPfyP6z4

I'm just not convinced how useful this will be. On one side you're very limited to the default style and list of available components. On the otger side you have to learn a new framework. It seems to me if you invest the time into a static web framework instead, you can design sites like this just as quickly while being able to dig deeper into html and css whenever you need to.

Acknowledging my ignorance in advance— what static web frameworks do you recommend for this?

Doing an A/B test on the Python source and the generated html I don't see much difference when I squint my eyes. Why drag Python into this?

I agree for simple examples there isn't much difference but I personally find this much easier to read and write: https://github.com/pycob/pyvibe/blob/main/generator/docs/abo...

than the output, which is this: https://github.com/pycob/pyvibe/blob/main/docs/about.html

It's true, I could hand code an HTML page but I never remember what meta tags to include, how to make it mobile responsive, or do things like set up a grid. I don't normally create front-ends since most of my work is in data so I'd have to look all this stuff up every time I come back to it.

I think the market for this is Python developers who want to build quick front-ends to something useful that's written in Python without having to re-learn or look up the front-end stuff.

JSX, Vue, or even django templates spoiled me so this imperative Python syntax sends my eyes spinning and I cant even think about reading somebody's nested loops. Has anybody got used to this syntax for complex layouts? Or has anybody made a templating language for this?

What problem does this solve? I mean, we have these kinds of frameworks/libraries for decades now...

people who know python but don't know (and don't really want to know) react / html / css / js, but still want to build a simple web app. kinda like plotly dash, streamlit, pynecone, etc - except without the web server part - just generates the html.

I'm planning to add something like this to https://inventai.xyz. Subscriptions aren't ready, so you can't actually generate anything yet.

Cool! I'd love to check it out!

Interesting. Cool to see python becoming more "full-stack"

hehe EXACTLY. python will take over the world!

This is cool. Good work.

Thank you!

What's the difference between PyVibe and PyCob?

Apologies for this confusion. Pycob was my original project that included the UI components as well as a web server, user auth, cloud services, and hosting.

I've decided to split the UI components from the rest of it because I've realized that they don't need to be bundled. The UI components are in a library called PyVibe.

Pycob will just be Cloud Services for Python Web Apps

PyVibe will be the Python -> HTML library

It sounds like you're talking about Jinja? Check out the includes / extends directives.

Also, look into HTMX.

Hey. I will try it in a upcoming hackathon!

Oh! What's the hackathon? If you're looking for teammates, I'd be down to participate depending on the project.

On of the best programs for linux tbh

Interesting. Does it dump the HTML/CSS and then someone needs to go in and add interaction?

So for now client-server interaction would be via forms. So you’d have a form, the user inputs something into the form, then that goes to the server as a query parameter. Then your endpoint would construct the appropriate response to those particular inputs.

For fully client-side interactions, the components are typically bundled with some JavaScript code to make that component interactive. So for example, you can include a datagrid, which has built-in sorting and filtering capabilities by doing page.add_datagrid(df)

I think if you need a lot of client-side dynamic capabilities, then that would be custom JavaScript that you’d either have to write or include a JS library from somewhere.

Is this a competitor to streamlit?

yes, it would be an alternative. imo it's a bit more powerful esp for multipage apps and uses more traditional web dev techniques (eg routes, etc). streamlit is tricky bc it reruns the whole script each time there's any user input. but it's also a bit more involved (eg you need to use flask or something to serve while streamlit has a webserver built in).

It's bit odd. You would want the form to be handled by Django forms system or wtforms in Flask - and having third party package to provice CSS for the classes they used (or you use on a wrapper element).

And you can make reusable components with Jinja or Django templates as well.

This is really really nice!

Thank you!

Reply on news.ycombinator.com