Clojure: Realtime collaborative web apps without ClojureScript | Hacker News Reader

Clojure: Realtime collaborative web apps without ClojureScript

andersmurphy.com

172 pointsbko34 comments

One thing I've always liked about the Clojure community, is that they are obessesed with state. I think they have correctly identified, that when things go sideways, it is because state management was involved in some way. This is a cool front end approach using datastar to put all of the state on the backend and side step front end state management entirely. There are some other really interesting things happening with electric clojure and a new "framework" called Replicant. Of all of them, Replicant seems the most intriguing to me personally. If it didn't exist, I think I would be trying to use datastar as this article outlines.

single minded react user signing in. When dealing with state in react theres generally a good incentive not to model state exactly like the database model. Does Clojure removing this abstraction improve the application? I can see many pros but not knowledgable enough to see the potential shotgun

The Clojure ecosystem embraced react early and built on top of it (with OM, Reagent and Re-Frame (for SPAs). The UI = f(applicationState) is definitely viewed as the correct approach. In other words, gather all your application state in one place. Whenever it changes, pass all of it to a single function to produce the next version of your UI.

Replicant takes this idea and runs with it, narrowing it down even further: The function is a pure function that returns the new UI as data. Replicant uses this representation to update the DOM accordingly.

That’s it. No partial updates, no mutable objects, no network activity from UI components. It’s basically a templating system. You give it data, and it returns a view. Whether that data comes from a DB or any other place, it's just a clojure data structure. Here is the article that most of this comment is lifted from: https://replicant.fun/top-down/

Doesn't re-frame also work with ui as a pure function of state? (https://day8.github.io/re-frame/a-loop/)

So, is the main difference that replicant is not built on top of react?

reagent (which you'd put re-frame on top of) is what handles rendering (via React) when we're using re-frame, and is the library you can say does "UI as a pure function of state". re-frame is basically like redux (over-simplification) but for the CLJS ecosystem, focuses solely on state and how to mutate it.

I agree with state management being the culprit. But the most-hyped solution nowadays seems to be: "We'll just ignore frontend state, whatever the consequences for user experience. We'll just say this is how HTML/CRUD was always supposed to work and that will make it fine. [Appeal to authority]"

Based on my 30 years of experience building software systems, state really is the biggest cause of problems and bugs in systems. The less state you can have, the better. If you can compute something instead of storing it, do so. Obviously you always eventually end up needing state, but it's good to minimize its use and treat it very, very carefully.

There is a direct correlation between complexity of the state model and the risk it imposes. State doesn’t have to be risky at all. If in an application there is only a single state object and it’s only a single dimension in depth it is never the pain point. In fact it helps to identify where the actual pain points are elsewhere in the application.

This is why I will never touch something like React, a self inflicted wound.

I use React in ClojureScript and I don't have problems with state. I treat react mostly as a rendering interface of sorts: my code is a function of state, and React is just a tool that does the shadow-DOM thing. I don't see a reason to avoid React.

There are two main high-level approaches for architecting web application client state:

1. Client-heavy state management (separate client application)

2. Client-light state management (thin client)

The state has to be tracked and managed somewhere. In most web applications, that state is persistable - at some point, it will end up in a DB of some kind.

As such, we already have to handle it on the server side, regardless of whether we also choose to separately manage it on the client side or not.

So the question at the end of the day is: Do I want to separately manage the application state on the client side, when I am already managing it on the server side?

In a lot of situations, the pragmatic answer would be "No". Why? Because managing the state in two places causes duplication, synchronisation issues, diverging logic issues, and so on. We are increasing the accidental complexity of our solution. Maintainability becomes slower, riskier and more expensive.

For some applications, the extra complexity might be worth it. In most web applications I have worked with (eCommerce, internal business tools, booking systems, online calculators, business dashboards), it is not.

And "let's keep all state and processing on the server" does not anymore mean no real-time or highly interactive features. With SSE and fast HTML fragment merging, we can enjoy highly interactive and performant web applications with almost no client-side application state management.

In eCommerce, saving the cart on client side and doing optimistic updates in case client network is unstable is a standard feature.

It still means less real-time and less interactive and likely more costly. Separating front and server is not a bad thing even if it brings some kind of duplication.

With .cljc files you can avoid the duplication anyway.

I've had quite my fill of "real time" and "interactive" web pages. Slow, bloated, terrible UX as often as not.

Even Datastar/HTMX might be too much. Server-side HTML can get you very, very far in most applications. It's faster, lighter, and looks and feels cleaner.

Case in point: do you really prefer a heavy, real time interactive message board like the default interface of Reddit? Because we're talking on HN, which is super fast, light, usable, and completely server-side.

Htmx is the exact opposite of what they're talking about. There are two concepts here: how you render and where you keep state.

HTMX is real time in the sense that it updates over the wire, but you're still keeping all state on the server side.

I don't want to reload the page to open a drawer. You'll notice HN isn't exactly mobile friendly because that requires hiding enough options that you need drawers or other ways to shrink and expand. You can do that with CSS, but that's still state, just state localized to that page load.

And there are sites that have unavoidably complex session state. You can have a resource that's on half the pages but is expensive to compute. I don't want to load it every time when almost none of it changes. I don't want to cache it on the server if it's a big payload

There's a world of difference between no state and over bloated SPAs

you might want to consider fixi, our ultra-minimalist implementation of general hypermedia controls:

https://github.com/bigskysoftware/fixi

The linked post relies on the Datastar project, which requires use of `unsafe-eval` in one’s Content-Security-Policy [1]:

> When using a Content Security Policy (CSP), unsafe-eval must be allowed for scripts, since Datastar evaluates expressions using an IIFE (Immediately Invoked Function Expression).

The project itself links to Mozilla’s docs on CSP, which state:

> The unsafe-eval keyword can be used to override this behavior, and as with unsafe-inline, and for the same reasons: developers should avoid unsafe-eval.

Out of the box, htmx uses a similar approach, but one can disable this use of eval [2]:

  htmx.config.allowEval - can be set to false to disable all features of htmx that rely on eval:
  
  - event filters
  - hx-on: attributes
  - hx-vals with the js: prefix
  - hx-headers with the js: prefix

[1]: https://github.com/starfederation/datastar/blob/develop/site...

[2]: https://htmx.org/docs/#configuration-options

Author of the blog post here. Happy to answer any question!

Oh and something I forgot to mention in the blog post.

- The server is in Europe. - The process can be modified at the REPL while It's running (and the changes get pushed out to all users by the SSE). So the board, HTML, CSS etc can all be changed on the fly without a restart.

My summary: one single copy of state is maintained on server side and clients are updated through polling or SSE.

Basically. It's generating the html for the game state every 200ms (regardless of the number of connected users) and pushing it down to any connected clients via SSE. The magic is Brtoli + SSE + a properly sized context window means it runs well even on 3G connections (you can play with this in chrome dev under the network tab throttle option).

BTW the demo app is broken: white screen with some Content-Security-Policy errors in the console.

It's embedded in an iframe on the blog, you can always try example.andersmurphy.com to access the game directly.

Sorry, I'm not a webdev so I'm not sure how to check this - but how large is the final downloaded page? (HTML+JS) The inspector in Firefox says it's 20MB, which seems large. But maybe I'm looking at it wrong

There's is not final download it's a streaming game. It's live multiplayer.

The initial bundle is 12kb (datastar + initial html + css). What you are seeing is the constant SSE connection streaming compressed data.

Are you behind a corporate proxy? Also what version of firefox? Because it's working fine on firefox for me?

To clarify, dealing with spotty connection / offline mode is not possible with this kind of approach?

Offline mode - not possible.

Spotty connection - no problem, SSE reconnects.

Reconnects eventually when things time out, but in the meantime, there will be a lot of frustrated waiting.

In my experience it reconnects faster and more reliably than web sockets. Also nothing is stopping you telling the user the connection is down/retrying etc.

Also because of the compression being used the game runs fine on 3G connections.

But.. SSE is limited to like 6 or so connections per browser, no?

Not with HTTP/2

With HTTP/1 the limit is 6 per domain per browser. With HTTP/2 the limit is negotiated between the server and client, 100 by default.

It's not a problem in practice, just ensure to use HTTP/2.

I'd also add even if for whatever reason you are on http1.1 (you should really use http2 for server driven apps there's a lot of other advantages). Datastar prunes connection based on the browser visibility API so on http1.1 a user would need 5-6 visible tabs/windows visible at the same time for you to hit any issues.

Assuming you are using Datastar the way I do with a single persistent SSE connection.

This feature is also much better for the battery life of devices and the server as you are only sending stuff if the user is there.

That's cool, but as a developer building web apps, I'm not competing with php websites, I'm competing with native apps.

Reply on news.ycombinator.com