The future of web software is HTML over WebSockets
alistapart.com
alistapart.com
We are in the dark-ages with respect to streaming sockets, and I am the architect of one of the worlds largest WebSocket and streaming services.
First off, unseating the operational benefits provided by request response is a huge challenge. It's possible, and I've done it. One of these days, my co-authors and I will present an OSDI paper with the broad strokes. It's very easy to list a lot of the challenges with scaling WebSocket, but each of them can be solved.
I believe we will have a nicer world with QUIC --> Http/3 and the up and coming WebTransport discussion. SADLY, that will result in a decade long "caniuse" issue, so we should expect to see WebTransport polyfills to arrive...
Second, HTML over WebSockets is nice, but what is better is reactive data binding over the socket. Let the browser maintain a DOM tree that templates over a JSON object, then reactive-ly update and it's an amazing experience. You have minimal data transfer, and minimal rendering updates. There is a great deal of power in having your UI be a stateless function over a giant JSON object.
Finally, the big challenge is dealing with the server state in a meaningful way. The exceptionally nice property of using request response for everything is the shared-nothing stateless nature of the web server. It's exceptionally forgiving in both operational and developer issues.
It's very easy to fuck up the implicit state tied to the connection, and this requires discipline to get right. It's very easy to ship unreliable software that then has issues based on the connection.
Now, I'm a HUGE believer in that we need to use streams for everything, but it requires a mind shift. The key thing to tackle is dealing with the server side state, and this is where I have my shitty idea of a programming language for board games. In essence, I am building a DIY database such that people connect directly to the database and get that sweet giant JSON object along with a stream of updates.
The DIY part is related to the fact that I invented a programming language to transact on the JSON object such that you get all the benefits of durability with the mental model of working within a single machine.
Totally agreed. I even pulled off an Elixir library as a POC to show this concept, here's an example project: https://github.com/surferseo/live_data/tree/master/examples/... (most relevant part of API is here: https://github.com/surferseo/live_data/blob/master/examples/... and here: https://github.com/surferseo/live_data/blob/master/examples/...)
It is indeed a very powerful and reliable way of ensuring synced state.
I'm making a chat bot thing (https://www.brian.bot/) and have added socket-based web chat... but I don't even know how to phrase the question of "how do I horizontally scale the socket connections".
Webchat <-> Server <-> Slack. Once I have more than one server dyno (heroku), then I can't ensure that the Slack event hits the dyno with the corresponding socket connection.
Thoughts / tips?
Your best bet is to use Amazon and then leverage their ELB offering with sticky routing. HOWEVER, now you have a new problem as capacity dies, gets removed, cycled, added, etc. Sticky routing was designed with the assumption that it was an efficiency play as opposed to a deterministic play.
What you will see happen is that your internal state will be split brain, and people joining the chat will not be globally consistent. Some people will be on host X while some will be on host Y. So, you have two choices. You could use a database to provide the global state, at which point now you have a whole bunch of problems. You can start with polling the database, and the key advantage of the websocket is moving the polling from client to server, so that will be a win for users. However, it will cost you. Second, you could introduce a message broker, but then you have the problem of how to initialize the state. Ultimately, everyone ends up with a hybrid database for storage and message broker for real-time.
In your model, the database becomes like Slack, and your Server becomes a proxy. This could be super neat, but what is the value-add of the Server?
The interesting thing to observe is that the solution is how to turn your front-end app into a database like thing. If the load balancer could route traffic like a database traffic controller thing, then you can put all your state in the front-end and move crazy fast. However, you will move so fast that you will fuck up your state. State is VERY HARD and exceptionally unforgiving to manage yourself.
There is great wisdom in using a database with versioning and all that. It’s a great abstraction having lasted so long. I’m working on the state bits and trying to figure out the people side of discipline with my silly language, and I have some novel contributions to provide over the coming years.
There are some linked pdfs on http://www.adama-lang.org/blog/some-thinky-thoughts-2021 which may provide some more insight.
Short term, I think I can get by with switching from Slack's events API to their real-time widget - aka, the server gets a socket to both the web chat, and the slack.
Long term - probably the Redis pub/sub with the socket-receiving server split out as a microservice (which normally I'd be "eh" about, but this seems perfect).
I currently don't have to worry about multiple people on the webchat (outside of needed features) and even once that's an important thing, I doubt I'll need to worry about exactness. As long as each message get delivered reasonably quickly, and to everyone, should always be good.
hmm, hmm hmm - great stuff, thank you!
When you need to send something through the websocket then publish a message via redis and have handlers in the code that are subscribed to that channel so every server can check if they have the websocket connection and whichever one has it will send the data through
Interesting that you're exploring this idea in the context of board games. I implemented[0] a similar idea while making a browser-based magic the gathering interface. Basically send commands to update the server state (a big JSON object), and the server does state diffs per client and sends lz compressed JSON patches. Worked quite well, but I haven't played with it for a couple years.
The core problem of web UIs has always been managing state. People like statically-rendered pages because most state gets kept in a) the URL, and b) transient HTML elements, and both are highly visible and easy to deal with in robust ways. When you're forced to have server-side state it's exceptionally painful, but people usually just avoid it altogether which in most cases pushes things towards the virtue of having less state anyway.
For complex UIs with lots of little interactions the above story doesn't really work, so we've invented state management systems for the client. Managing complex state still sucks, but it sucks less with these tools.
Sounds like this approach is just moving that state store back to the server, with a greased pipeline so it isn't so horribly painful to add new pieces of state even though both sides are involved. But you'll still face all the same problems you face with a front-end store: transientness, cache-clearing, even modeling is hard when you're dealing with a big blob of mutable state. And there will be the additional challenge of associating this stuff with a user (which you get for free on the front-end, even with modern state stores).
> Let the browser maintain a DOM tree that templates over a JSON object, then reactive-ly update and it's an amazing experience. You have minimal data transfer, and minimal rendering updates. There is a great deal of power in having your UI be a stateless function over a giant JSON object.
> this is where I have my shitty idea of a programming language for board games
It's funny, I started building a web-based board game to play with family during quarantine, and I landed on this exact approach totally by coincidence :) I'm using polling requests instead of websockets because I'm lazy/it doesn't need to scale, but it's the exact same idea where the server just publishes a giant JSON object of state and the clients are pure functions of it (and send messages to the server to mutate it). I wonder if there's some aspect of board games that pushes one towards that line of thinking.
I like this approach as well, but I do worry that users today expect optimistic updates on the client, and the injection of a X00ms delay would make the app feel sluggish.
I think the key thing here is figuring out how the UI can be made mostly declarative and a stateless function of the state. I'm drawing some inspiration from
https://flutter.dev/docs/development/data-and-backend/state-...
and the challenge is the polish to make products great. My strategy is that I'm going to make a UI editor so I can make faster progress on designing the board game UI and then build a minimal set of controls to do data binding. The key is to get into the fight about what is the polish needed, and what reasonable escape hatches are needed.
> I wonder if there's some aspect of board games that pushes one towards that line of thinking.
There is, they are exceptionally complicated!!! SO COMPLICATED! The key is the implicit state machine between all the players, and you just can't quickly model it with traditional tools and databases. All it takes is a single card or game piece (which changes rules) to invalidate EVERYTHING. Games are exceptionally messy!
Thus, my processes clearly has to sustain 100k+ connections. And, at a moment's notice, I should expect the process to go "poof". Maybe someone is going to do a kernel upgrade or someone decided to unplug a machine, I don't know!
Having operated with the above assumptions, I found it hard to be wise to avoid that and then invented a programming language that would operate in a modern cloud environment.
I need better hobbies, or not... who knows.
So, XSLT for JSON basically. I like it.
The UX back then was horrible, however technology has come a long way since.
Perhaps you or someone in this topic might be interested in my little project[1] fit for innovative ideas like this. It is not very far from being available for experimental use in place of JSON. A future direction is experimenting with replacing HTML[2].
It sounds like the one I was working with ~18 years ago.
Except it used HTTP (long polling or comet) instead of HTTP/3 for the transport. And more partial rendering to DOM-diffs on the server side because it was faster than doing it all on the client, but the client still applied diffs.
The issues with full DOM state synchronisation, handling network errors and recovery, transport of templates and data, minimisation of transport subject to maximising responsiveness, some amount of anticipatory prefetching, were much the same then as now.
A lovely thing about all this stateful streaming cleverness is you end up building a mostly stateless, functional-reactive model on top of it again, that ends up extremely robust.
It's just faster, and it can react immediately to server side data changes as well as client side. And that model can be declarative, it doesn't particularly need to be in JavaScript despite running on the client.
Then other browsers copied it, adding their own quirks. The XHR standard wasn't created until about the same time as IE8 was released. So it's not possible for IE before IE8 to break the XHR standard, as there wasn't an XHR standard before IE8.
Long-polling is still used as a fallback today. The modern alternative is WebSockets, but they do not work reliably in every network environment, either at the client or server side.
Last time I looked, Facebook and Gmail were using long-polling to good effect.
Despite the conceptual ugliness, long-polling when implemented properly works quite well and is extremely reliable. In practice, "quick and dirty" long-polling is not always implemented well, but it's not hard, just takes a little thinking about unreliable networks.
XHR has nothing to do with long-polling particularly, and it doesn't remove the need for long-polling either. Long-polling is usually done with XHR these days, but it can be done easily without XHR, using invisible iframes or dynamic script elements. The latter is called JSONP and you've probably heard of it. SSE can also be used.
All of these things have nothing to do with HTTP/1.1. They work perfectly well over HTTP/1.0, 1.1, HTTP/2 and HTTP/3.
Isn't that Firebase DB?
The key thing that I can do is await on players decision, so I can block the entire process for multiple people to do things.
I think a lot of the "best-practices" for SPA is trying to do this. But as streaming the data transfer is not yet common, the data is abstracted as a giant JSON object to be a parameter of the UI as a stateless function (React+Redux/MobX/etc, Vue+VueX). When enforced correctly on the project level, what's complicated _is_ syncing the data to the server. I think you'll like Supabase's subscription approach to reactively bind the JSON for the client.
I think a separation between the info-web (document-content heavy) and the app-web (needing stateful data sync) will be more clearly defined in the next few years. Nobody likes trying to sync data between the client and server over HTTP calls; we just have to wait for streaming to be the norm for the app-web (and native apps, ofcourse).
This is the approach we are building in https://braid.org. We are extending HTTP from a state transfer protocol into a state synchronization protocol, so that when you do a GET request, you can also be promised to receive all updated versions of that resource. This extends very nicely with software that binds the resource directly into a DOM element.
It appears to work well. Security was tough work though (at least I thought it was). And scaling up, no matter what tech you have access to, can be costly.
Also, determining how to handle disconnections is quite an interesting UI problem. Auto reconnect might be an option, but then you need to think about a retry policy. And it can be harder than you think... Mobile phones drop connections with surprising frequency, so it's worth thinking about very early on.
The scaling up is a challenge due to the lack of common investments. It can be made absolutely cheap, and it is way cheaper than polling.
People that dismiss websockets have a point that it throws away decades of wisdom, but the winnings and potential are there. The challenge is curating the right ideas and educating everyone... is hard.
Only state shared between multiple clients should go to the server.
In server-rendered app, there's nothing like UI state.
There's a request. This request and the database is enough to build response. There is no UI state. Your app can be (and often is) absolutely stateless and restult depends only on request and a state of the storage.
While when you have SPA, API & server, you lose the application state and it's not possible to store the whole app state in the client for many reasons (security, scalability, synchronisation...). So you have to store subset of it. And you have to build a machinery that will do all this synchronization so your states stay in sync. It's double the effort to get 5% improvement.
Of course there is.
For example, any state related to interactivity like forms, client-side validation, widgets, filters, etc.
Basically, I'm happy with my latency (for board games) which I hope to migrate closer to people as more edge compute offerings emerge. However, I'm wondering what the path is to find some of these ideas in more latency sensitive applications.
The explanation for how the net-code behind Overwatch is a good explanation: https://www.youtube.com/watch?v=W3aieHjyNvw
and I'm just trying to figure out how to build the reactive tree in a way to support it sanely.
1) Nothing bad can happen if we show some optimistic result before server returns, but server side will end up in different result.
2) Something bad can happen. Accordingly, for (1) we can do optimistic update, for (2) can't
Think about the nature of a document store, you put data in and data out. You gain concurrency by using compare and set such that multiple requests can compete, but this is expensive. First, you risk conflicts which require re-execution for the losers. Second, you must download the entire object, make a change, then write the entire object back.
The key to what I am doing is that I'm having the language take a message from a durable queue, integrate that message into the document via some code, then emitting a state change out. The client then subscribes to the document's parts to updates its UI state, and as the document changes, so does the UI.
The UI can be much more complicated while the server can be beautifully simple.
Have you used HTTP/1.1 comet-stream? I recommend you look at this sites source before you commit further: http://fuse.rupy.se
I agree with this and it's why I believe that Phoenix LiveView is the best solution to this problem. The programming model and tooling that you get with the BEAM (along with things Phoenix provides like PubSub) are the best there is to manage this complexity.
biff implements this as a framework, backed by the Crux bi-temporal DB. It natively uses EDN rather than JSON, but emitting JSON should be trivial if that's needed.
Something like this: http://jasonette.com
It's sad to think how much effort is spent reimplementing technologies because the IPv4 address space wasn't large enough.
If IP address space were larger, we wouldn't have needed to rely on Network Address Translation (NAT). NAT became a poor-man's firewall and effectively blocked peer-to-peer connections making it difficult to host a server at home. This incentivized early ISPs to offer only asymmetric connections that pushed users towards a consumer-only service that crippled early Internet protocols. With the World Wide Web being "the" reason people got on to the Internet in the first place, early NAT-enabled routers focused only on HTTP tunneling, so completed protocols that required inbound connections were finally killed off.
Who would have thought that a small IP address space could result in a mass protocol extinction event from which only HTTP and HTML would survive....
Most ISPs (with the exception of very small ones) until recently assign one public IP address per customer and NAT is done on home router. With this setup it is not hard to host a server at home - it is just one rule on router to NAT all incoming connections to that server. Many SOHO routers support such setup. Alternatively, the server can just work as a router.
> This incentivized early ISPs to offer only asymmetric connections that pushed users towards a consumer-only service that crippled early Internet protocols.
Asymmetric connections are offered because many last-mile technologies (like ADSL and cable modem) are asymmetric on link-level - they use different bandwidth allocations for upstream and downstream.
HTTP is successful partly because it's simple and "stateless", in the sense that all protocol state is held by the client in cookies. Keeping all state in cookies+database provides some useful properties, in particular around scaling, being unfazed by IP address changes (e.g. wifi to wired), reboots of both clients and servers, version changes, hibernation/wake-from-sleep, and a variety of other things that otherwise break connection oriented protocols.
Of course sometimes people screw that up in other ways, hence the popularity of "clear your cookies" as a troubleshooting tip. But overall it works pretty well. Look at the prevalence of hacks on top of SSH to create persistent sessions, to see what has to be done when you don't have that model. SSH only survives as a protocol because remote administration doesn't need to be a particularly great user experience and it doesn't need to scale, so legacy and inertia wins out. Otherwise we'd probably see a request/response protocol take over there too.
X11 was always a PITA not only due to NAT but also due to institutional firewalls, e.g. universities often have enough IPs to give everyone a public IP but firewall traffic anyway to stop exploitation of vulnerable devices. So the whole "remote client connects to local server" model was not only confusing but also a poor fit for the real world internet in which centralised, professionally run servers are invariably more secure than scattered non-professionally run edge devices.
Anecdotally, the typical single Rails server process seems to be perfectly happy supporting nearly 4,000 active connections. And you can easily swap in the excellent AnyCable to bump that up to around 10,000+ connections per node by not relying on the built-in Ruby WebSocket server.
That's not a lot of connections on it's own. If you are implementing a chat service you'll still need some form of pub/sub (or something) to glue all of that together when users connect to disparate nodes.
Also
How about client-side validations? Easy. On every input change, round up the form values and send ’em down the WebSocket.
That's just server side validation.
As a disclaimer, I haven't used any of the HTML over WS frameworks the author discusses, but I've spent a fair amount of time dealing with scaling socket connections. It definitely makes the question of scaling horizontally more challenging. I feel like this article really glosses over that aspect.
This "solution" completely ignores the purpose of client-side validation which is to a) reduce load on server-side resources, b) validate faster without the network latency and c) validate more securely and privately by not sending the content to the server.
Just the other day someone posted on the Elixir forums about live_redirect causing 2 network round trip connections[0]. Basically there was double the latency to transition between pages due to this "bug". I air quoted "bug" because it's working as intended, it's just not optimal.
The creator of Elixir mentioned it was doable to fix the issue but when an issue was open on GitHub[1] it was shot down with a "this is a known tradeoff, you'll need the 2nd route trip for the websocket". I'm not sure what the state of the issue is but it currently stands as closed.
I've pinged 80ms to servers where LV was being used and the delay was extremely noticeable (before I even knew about this double network round trip issue). It feels much worse than a Turbo Drive / Turbolinks driven site (which uses HTTP instead of websockets like LV does). In some cases LV feels slower than a regular site even with the DOM having to re-parse all of your assets. The only time LV feels like a net win to me is when you're on a local connection with 1ms of latency.
I wanted to use LV a lot, but backed out because I kept running into bugs and missing features. Plus after feeling a LV site on a non-local connection I can't say that I would want to impose that experience on users. Especially folks who happen to connect from let's say Europe to the US, or even further away. The web is a global place.
[0]: https://elixirforum.com/t/shortcomings-in-liveview-are-there...
[1]: https://github.com/phoenixframework/phoenix_live_view/issues...
I gave it a fair shake a few times.
Once when it first came out, then again a year later and then again 6 months ago.
Lack of hooks wasn't a concern I had at the time. It was more around core behavior of the library and critical features that were missing. Some of those features have been added after I posted about them but all that did was destroy any confidence I had in using LV because these are things that would have been encountered on day 1 of deploying a single LV app to production. We're talking huge things, like how to invalidate and update assets in the <head> of your page in a user friendly way.
It left an impression on me that LV isn't really being used much in real world apps by the team developing it. I could be wrong of course but that's the impression it left. Plus it feels like it's taking a really long time for certain features to make their way into the library. For example file uploads took something like 18 months to go from being talked about publicly to getting an alpha release and now it feels like it's on the burden of the community to test this in production without really knowing much about the feature.
That and the docs still leave a lot to be desired (especially around the time I read them) and the story for the last ~18 months is that since LV is a pre 1.0 release the docs aren't really written up yet since stuff is changing all the time. I know docs take a long time to write (I've written literally over a million words of blog posts / course notes / documentation) but docs and practical examples are also the most important thing IMO to nudge folks into using something.
Personally I don't want to have to read minimal docs, API specs and dive into the source code that I can't read very well because there's a lot of macros to see how something works just to use it effectively. Especially if I'm on the front lines of having to pioneer the tech, which means I'll probably be under pressure to report and fix bugs that I don't know how to fix.
I don't know. All of this experience with LV and Elixir / Phoenix really made me understand that this tech stack is not for me. Especially not when Hotwire Turbo exists and works with any back-end language and it also has proof of it being used in a mission critical massive SAAS application (https://hey.com). That leaves me super confident that it'll work for me (and it has been), even outside of Rails.
Maybe in 5+ years I'll try Elixir again (hopefully Stripe and other payment providers have Elixir clients by then!), because the core Elixir eco-system in general has a bunch of nice things. It just doesn't feel optimized yet for building applications (IMO). At least not compared to most other web frameworks.
Also, while I don't use Laravel I also have major respect for Caleb Porzio. He created Laravel's version of Live View (LiveWire)[0] by himself. It mainly uses HTTP and he also has a ton of docs / videos on implementing practical application features with it. He shipped 2 major versions and everything about its API and docs just oozes creating something made for developers to develop applications. It's funny how a slightly different position on something can make something explode in popularity.
I haven't even written a single line of Laravel and have no intention on switching to it, but his presentation and execution of an open source library is something I admire.
LiveWire builds on Laravel which is a massively popular framework on a massively popular language, Laravel itself using components from Symfony that is basically the backend framework with the most contributors in the world.
But LiveView may hit 1.0 this year :)
For comparison's sake:
- [LivewWire] Caleb (creator of LiveWire) made 1,000+ commits and added 200k lines of code from Jan 2019 to Feb 2021
- [LiveView] Chris (creator of LV) made 700+ commits and added 80k lines of code from Oct 2018 to Feb 2021
- [LiveView] Jose (creator of Elixir) made 450+ commits and added 25k lines of code from Oct 2018 to Feb 2021
There's even more contributors to LV (207) overall than LiveWire (145) which is interesting because LiveWire is 3x more popular based on GitHub stars.
From that you could say that LiveView has more manpower than LiveWire since Caleb wrote all of that code himself in a shorter amount of time while 2 people are main contributors to LV.
Plus at the same time Caleb wrote a huge amount of practical documentation (focused on building features), created lots of screencasts where you build features you would expect to see in most apps (data tables, etc.) and started a podcast around LiveWire. And on top of all of that he created AlpineJS at the same time.
> LiveWire builds on Laravel which is a massively popular framework on a massively popular language, Laravel itself using components from Symfony that is basically the backend framework with the most contributors in the world.
At a fundamental level it feels like the creator of LiveWire is investing in creating a tool that helps developers build applications better and faster. I think that stems from Laravel giving off a sense of developer productivity for building apps, but don't forget that library creators are making these decisions.
I never really got that same impression from working with Phoenix or LiveView. I think it caters towards a completely different type of developer than myself which is why I struggle so much using it. It always felt like instead of showing you how to do something, it makes you figure it out yourself.
> But LiveView may hit 1.0 this year :)
That would be nice to see but I'm not sure how much of a difference that will make in the short term. In the short term going 1.0 is really just deciding to tag a commit. If Chris and Jose plan to write another book to use LV that could still be over a year or 2 out unless they've been writing it in private, or if they re-write the documentation that's also another long journey.
Neither of them really strike me as the screencast type either. When they do create videos they do an excellent job at explaining things tho. Really wish they did more of them to be honest. But yeah, these things take time. I don't know what their schedules are like too so maybe it's not even fair to compare LV vs LiveWire in terms of how fast the library is being built. Maybe Chris and Jose only work on Phoenix and LV for 2-3 hours a week where as Caleb is working on his stuff full time.
José works at the same time on Elixir, Phoenix, LiveView, Ecto, Nx, Dashboard and more, it's a lot.
To be fair there also has been integration of LV in the Phoenix mix tasks, in addition to the Dashboard (using LV, ah!), so it's not like things are stalling.
Maybe Chris and Jose are more efficient? Maybe the Elixir/LiveView code is just better written and doesn't need rewrites?
Why do so many people glorify quantity over quality?
We are dealing with limited information here, and GitHub makes it easy to see at a glance what folks are doing on a project. It just so happens it focuses on presenting commits and lines of code written.
Everyone knows it's not the best metric but to get a high level overview of activity those stats work. Especially when the person I was replying to said LV is maybe moving slow due to a lack of manpower. Those numbers show the opposite (LV has more folks working on the project than LiveWire, despite it being much less popular due to Elixir being a smaller niche than PHP).
> Maybe the Elixir/LiveView code is just better written and doesn't need rewrites?
I don't think it's fair to jump to any conclusions about the quality of either code bases.
I would care a lot less if we were talking about billionaires building commercial startups, but you’re literally attacking open source contributors. Moreover, my personal experience in using LiveView and interacting with the community has been the exact opposite of yours, so I find your comments to be totally misleading for people who might think that your walls of text have any nuggets of wisdom in them.
I’ve tried a lot of open source software that left me thinking, “wow, what a waste of my time.” For some reason I never felt a need to openly attack the authors of these low-commercial-value-for-the-creator, volunteer-driven projects. You might want to consider directing your critical eye toward the actual problems in this world, rather than volunteer programmers’ projects that didn’t met your personal, highly-opinionated requirements.
The parent comment is Exhibit A for why so many people refuse to get actively involved in the open source software scene, and why so many drop out.
Perhaps I was too critical in some of the replies and should have phrased things in a more positive way but I did use Elixir / Phoenix / LV on a daily basis for a pretty long time.
Over the course of those 18 months I'd estimate putting in 250-300 hours of programming time in spurts while writing 3 applications totaling around 9,500 lines of assorted Phoenix / LV code. Not a lot of code by any means, but enough to put a decent dent towards developing the main app I was building which is where I encountered those issues at the time. I have no agenda or reason to make anything up. I want to see Elixir and Phoenix succeed in the end. I just decided to temporarily put it on hold until it gets over the early adopter phase.
Please Nick, you can literally check your links to see this is inaccurate. I mentioned a solution after the issue was closed - and not before as you describe. And I figured out the solution *with Chris*, as I clearly mention in my comment.
And then later on:
> dive into the source code that I can't read very well because there's a lot of macros
There are like 8 macros...
> since LV is a pre 1.0 release the docs aren't really written up yet since stuff is changing all the time
Seriously? No one has ever said this is the case. Just go to the [official docs](https://hexdocs.pm/phoenix_live_view). All public functions are properly documented, there are introductory guides, etc. Sure, we don't have official screencasts but that's something we rely on the community to step in: Pragmatic Studio has a fantastic course on LiveView (which I was involved as a sounding board), there is Grox.io, etc. And it hasn't stopped either, a new book was literally announced today.
There are other inaccuracies in the comments below but honestly I don't have the energy to go down this rabbit hole again.
I should have used quote / unquote instead of the word macro.
When I looked into the code I started with the engine and renderer. Between both modules they had dozens of quote / unquote usages which to me was hard to follow. Not because it's written poorly or anything like that, but it's not exactly easy to trace that code to learn how something works in more detail.
> I mentioned a solution after the issue was closed - and not before as you describe.
Yes, after it's been closed. But look at it from what end users of your library see from that chain of events:
1. User asks question on forums and presents a case where something very bad happens (2x network round trips)
2. User posts issue on GitHub
3. Creator of LV says it's a known trade off and quickly closes the issue
4. You and the creator of LV talk offline and figure out a potential work around
Re-opening the issue after #4 would have done a lot of good because it shows at a glance that it's a current issue, it's being addressed and open for discussion.
With the issue being and staying closed this gives off a message that you're not actively working on fixing the bug and aren't open to any form of discussion or assistance around fixing it. Maybe that wasn't your intention but that's the message you're sending to some people.
> Seriously?
That's the answer I've always received in the past when asking questions about the state of the docs on Slack and IRC over the years. The current docs usually give you a partial understanding of how something works. It's usually enough to get a basic idea of how something might work but not enough to get the ball rolling to implement a solution in your own application. I don't think I'm the only one who feels this way either because I've seen a lot of repeated questions on IRC and the forums, especially around LV components.
( Results were from 2016, so may be things has improved )
Where Rails with “live view” tooling would work really well is building CRUD heavy sass apps that don’t need a persistent connection, just want really fast page refreshes and/or partial updates.
The work the basecamp team did on Hey seems to be a good illustration of that.
He describes a "please wait buffering" future of web software. No thanks.
> Do we really believe that every one of our users is going to have a device capable of digesting 100 kB of JSON and rendering a complicated HTML table faster than a server-side app could on even a mid-grade server?
Yes. This really isn't hard. Pretty much any smartphone can do this with blissful ease. And it scales horizontally because your server can just worry about getting stuff from the database and handing it out.
What it doesn't do so well is lots of parallel requests and responses out of order. HTTP/2 is better for that.
Both versions will close the connection if it's been idle for a while, and automatically reconnect as needed.
Latency seems to have been conquered.
I dare you to say that from Australia.
But if you have a server in Australia then I would still say that.
I think there's value in keeping our protocols as simple as possible to accomplish the task at hand.
You may be confusing it with HTTP/2 Push, where a server can inform a client of a future resource it will need, which is _de facto_ dead, with only a few real implementations beyond the bare minimum required by the HTTP/2 specification.
I guess I'm not seeing what the alternative setup would be, even with WebSockets.
For infrequent requests everything goes through the api, but for stuff like chat and live-document-editing, nodejs is the better solution.
But to answer this new question:
* You can have bidi without WS.
* Because WS is more complicated.
Because I was answering this question: "What's wrong with normal HTTP requests for client->server streams?"
> You can have bidi without WS.
Only by combining HTTP (which doesn't scale because my api server runs rails) and SSE (which does scale). If I want scaling bidirectional communication, I need websockets.
> Because WS is more complicated.
More complicated than SSE? I don't think so.
But today I always start with plain HTTP, then try SSE, and finally WebSockets if I have to.
No diagrams. I don't see much of a comparison between costs/benefits of alternatives.
Nice hypothetical. I didn't read it, because I've seen the format before and I'm getting a bit bored of it.
Whereas if it cited some other engineers/technologists, some studies, some academics, etc., then perhaps I'd be more inclined to read this.
But one person's opinion? Nah. I need an aggregate of opinions to be persuaded to read another "The future of thing..." article.
When the Trough of Disillusionment hits, us old farts pop out of the woodwork to say "see, told you so" whether we were right or wrong.
I think it also relates to operational professionalism and having your tools and toolbox in order ala Adam Savage and Jamie Hyneman from Mythbusters. They both had very interesting relationships with order, process and tooling.
I think structuring dev teams around a professional kitchen is really powerful model. All roles are critical, the sous chef, the busser, the prep cook, the dishwasher, everything.
Though kitchens are especially interesting as they have pipelines, surges, lots of tasks that can be broken down into a series of steps.
Mythbusters has another interesting aspect in that they came from movie production background, so wall time was really critical, you can't hold up a shooting day to get a better glue formulation for simulated flesh falling off bone.
Part a thing is its structure, but the other part is how it evolves over time and its reaction rates to stimulus.
Agree on the beers.
format: blatantly unoriginal
content: not persuasive, underdeveloped
effort: very low
value: very low
"The future of" is a very bold statement. The more cliche it becomes, the more I expect an author to actually outline, in a bold, cohesive, relatively comprehensive way, their theory.
Otherwise stop trying to tell people what the future will be-- it's a waste of time & attention.
Sure, the headline is catchy. The rest seems like the author was simply trying to reach a certain word count and claim their cheque.
--> "Future of" articles should read like a graduate academic paper-- give the future of technology the proper respect it deserves.
--> Or, they should be a policy paper.
Not a pop tech piece. Pop writers-- stop with the silliness-- stop appointing yourselves as definers of the future unless you expect folks like myself to come along and bring you back to reality.
____
Take this linked citation for example:
"Fast-forward to 2020 and the web isn’t getting any faster"
It's linked to an article by an individual who has an undergrad degree in psychology, a grad degree in IT, and is a design/UX-related employee. That's not the person I would go to for a technical, knowledgeable analysis of internet speed evolution.
Why does it annoy you so much?
It results in devolution of knowledge and a disrespect of the attention of intelligent people.
"he could be right" -- what a waste of a statement unless it is followed up by any premise. He could right... Ok, that's a claim. But based on what?
The fact that it’s touched a nerve is more interesting to me
But if I write an opinion-- that's not ok. But you-- you're allowed to critique my opinion.
_______
Let's apply your logic to your replies:
Well, jack_riminton, It seems like I've touched a nerve.
Why did you feel compelled to comment on my commentary, when you could have just ignored it?
Why are you critiquing my opinion if you're making the case that opinions should not be critiqued (and should simply be ignored if a person is in disagreement)?
______
Ohh.... Now I see. You're the one who posted the article.
Well, everything is very clear now: Simple bias.
______
Note: You never followed up when I questioned the premise of your claim "he could be right." It seems you're not interested in analyzing whether or not he is right. So what's the point of even posting this article?
Yet the purpose of my entire thread is to draw attention to the fact that him being right will not result from conclusions of a poorly contrived, low effort, pop-tech article.
Secondly I don’t even agree with it
It’s an interesting (and apparently provocative) prediction. I’ve enjoyed reading the responses
Unless of course, showing the work was the task (such as writing out a proof) versus a straight computation.
He could be right because... (these are just generic examples)
- He mentions renowned research by XYZ who has through years of painstaking research concluded (as cited in XYZ peer reviewed journal) that the basis of his argument is sound
- He's an expert in XYZ field, having worked at XYZ company which clearly demonstrates proficiency in XYZ subject material
- He clearly elucidates, citing several researchers and with clear, understandable diagrams how the phenomenon XYZ works
... Yet you make no effort to construct a premise to base your claim upon. Because no real premise exists: the article's author makes no effort to support his own claim about the future.
The problem is that stateful connections suck and create artificial complexity when you don’t need them. Instead of managing the state of your app you’re now managing the state of your app and a connection to a remote server. HTTP is a stateless protocol on top of a stateful socket connection. It wasn’t designed this way by accident.
I’m all for new engineers building SPAs creating exciting new libraries, but HTML-over-websocket has been rehashed at least three times over my professional career so far and it’s not even that long.
Don’t get me wrong — websockets have their use cases. Multiplayer online experiences pretty much require them. It’s just that HTTP isn’t fundamentally broken, and by engineering HTML over websockets you’ll almost universally find yourself building protocols to abstract the state away... re-engineering HTTP in the process.
It’s certainly a fun project which I have personally worked on. But I’m not sure it has legs outside of that. Open to being proven wrong, the authors of HTTP aren’t infallible.
- What happens if the client’s internet connection temporarily drops? How do you handle client reconnections and retries?
- What if the remote server crashes through no fault of your own (AWS server rack catches fire)? How do you ensure a consistent reconnection that preserves application state?
These are just two questions that HTTP + Browsers solved decades ago.
Websockets are this cool technology that seems powerful and exciting until you realize the “boring old stateless World Wide Web” was built over decades of iteration using stateful sockets as the backbone, did most of the work for you in tandem with browsers, and is ultimately a lot cooler and more sophisticated than a lot of us give it credit for.
Imagine a few customers open up your site and leave for lunch. Or maybe just a random crawler or bot sitting watching for some piece of information they are scraping.
These sorts of issues can be challenging with stateless connections. I can only imagine how daunting they are with persistent connections.
Games basically do this routinely.
With server-fed, event-driven pages, you could potentially force refresh.
I assume web-sockets could make it work more robustly.
What happens if a webpage is loaded over plain old HTTP and then over lunch a breaking change is deployed. The page the user loaded before lunch is now no longer compatible.
It’s stateless. If the api changes from underneath and you’re polling or something, then just show an error, have the user refresh the page and move on. Or use any of the various api version mechanisms for backward compatibility. It’s not an easy problem but it’s a fairly well understood and managed one. Solving this kind of known problem isn’t, to me, a very compelling case for HTML over sockets.
I should probably already know this.
HTTP is quite literally "hypertext over sockets" so "HTML over websockets" is functionally equivalent... with a layer of abstraction added on top. If you build web software by sending HTML over websockets you're just reinventing what already exists; it's probably why it's never taken off. I think most developers realize within ~weeks to months of working on a "websocket-everything" framework that they're doing more work, not less.
A crash or someone hitting refresh on his browser is not that different in this respect.
WSs aren't a magic wand, they just allow you to do more with the same.
You basically end up with a half-assed, buggy, and potentially insecure reimplementation of HTTP running over web sockets running over HTTP at that point.
Are you going to do session management and identification by issuing a unique identifier as well? Is that unique identifier going to be valid in perpetuity or have a an expiry date? Is it going to be valid and secure across all sub-domains or will it be prone to cross site scripting attacks?
Because you can do things stateless HTML alone cannot do.
No one is trying to re-implement HTTP, or even remove it from the equation.
What people are trying to reimplement (or rather, replace) is the HTML + JSON:API + React + Redux mess. Websockets is a means to that end.
Unfortunately, great intentions alone won’t make this work; someone has to actually answer the hard questions about how it’s going to work too.
All the answers I’ve seen so far trying to answer them have been vague hand waving...
That doesn’t leave me with much confidence.
Here is the documentation for Phoenix channel authentication (what Phoenix Live View is built upon): https://hexdocs.pm/phoenix/channels.html#using-token-authent...
Additionally, here is some documentation on the security considerations specific to Live View: https://hexdocs.pm/phoenix_live_view/security-model.html
Searching can probably find you more information.
People did better than that, they already built applications with this model.
The purpose is in the things explained well in TFA: single source of truth, no replicated state, simpler development, and so on...
Who said the purpose would/had to be in handing "the server caught fire" scenarios?
It works for certain things and not for others. So, it's not "the future of web software" like the title says, it's one additional possibility for when it makes sense.
Why would you? Nobody seems to ensure that with regular SPAs anyway...
Here is a short demo of the product: https://www.youtube.com/watch?v=TwkgGBrK_tA
It uses websockets & JSON to connect the SPA UI (in Vue.js) to the app (in Go). That works well, and I can't imagine any advantages to generating HTML with Go, except perhaps less JS code :-)
It's possible but extremely difficult to create a "modern" experience with just html. Just like how we don't build houses with mud-bricks anymore even though it's easier than steel and wood.
"Modern" experience is not necessarily the same as "good" experience. Good old server side rendered pages are fine; SPAs are being built just because "we can", not because they are actually needed (sure, in some cases,SPAs are needed, e.g., chat sites).
I have yet to see a proper web app (not a blog/collection of static pages, but something that does something complex like let you manage inventory, do payroll, do your taxes, monitor sensors in real time, interact with geographical maps, or edit documents) that is done without JavaScript and has even OK UX.
SPAs are great because of separation of concerns. Sever takes care of the data (storing it, syncing it, authenticating your access to it, serving it, sharing it). Client takes care of presentation. I build SPAs because they are better. Imagine if all the apps on your phone or computer were effectively HTML documents rendered on the server. Emacs over HTML/CSS? Minesweeper? World of Warcraft? Would that be an OK experience? Probably not because you don’t want to wait for a page load every time you do something in your Minesweeper game or check stock prices or whatever. Why should web apps be inferior?
When people complain about SPAs it’s because they don’t like web apps running in the same environment as their document-based content. I guess some people prefer the ActiveX/JavaApplet model, which to a degree I can understand. Slack is an application, not a website. But let’s not overlook the fact that implementing a Slack client with just HTML and CSS would be miserable to do and miserable to use.
Inferior compared to what?
I'm not going to call your statement nonsense but unless you have proof that SPAs make up the majority of the websites/web apps online today, I'm afraid I must disagree.
Frontend dev is a hot mess of garbage JS cobbled together to make a psuedo native experience.
> but something that does something complex like let you manage inventory, do payroll, do your taxes, monitor sensors in real time, interact with geographical maps, or edit documents) that is done without JavaScript and has even OK UX.
This is not what the web was built for. The same things you mention dismissively (server side Static pages) are the core and norm ,by and large, the vast majority of what constitutes "the web"
Good, lightweight native apps can do all the above without breaking a sweat, and you can have multiple native clients for things you care about. I don't always have an internet connection but I still want to type my documents. I fire up word. Not google docs. I need to do some serious data wrangling - I fire up excel not google sheets. Need to do some networking for my router, wireshark. Need to read an ebook - adobe acrobat. Need to write some shader code - Sublime text.
So the way it looks to me that SPAs are trinkets - not at all mission critical beyond loading cat pictures faster or bending the web backwards to do things it wasn't supposed to. Server side static pages rule the web.
Complexity is just a good way to justify a large paycheck.
The second argument is that there isn’t a technology today that supersedes the web in one crucial feature: no installation required and updates are transparent. Various app stores are trying to solve this in a platform specific way but do you really want to live in a world where to publish an app you must include Apple and Google and Microsoft in your deploy path? You and I may be able to go to GitHub and download OpenStreetMaps code, then download all the data, install it all locally (most of the documentation last I checked was still in German BTW), and then use it to find directions to the nearest Starbucks. An average user will give up before spelling out “GitHub”. If you can point at any platform that lets me type in the name of an application and stat using it in under 1 second without installing anything whatsoever on my system, I will eat my hat.
That isn’t to say that we can’t have nice things. With time and effort you can create such a platform and get all OS vendors to support it and all the developers to move to it. But I will postulate that fixing the JS frontend mess will take at least an order of magnitude less effort. Therefore the web is where these types of applications will live.
Again I am not talking about things that are static. This isn’t about a blog. This is about whether a real application is better server-rendered or as an SPA and I argue that in 99% of cases an SPA will be a cleaner architecture AND provide better experience to the user. The fact that coding one up is annoying (arguable at best) is a temporary tooling problem, not a fundamental design flaw.
Top Gear proved it was well and truly possible. It's a popular mode of transport. It carries things. It has an engine and a mostly metal body.
It still isn't quite a boat
Web apps can walk and talk like native apps, but they aren't. Sure one off things like a Facebook or spotify or meme generators can rightfully be served from a web app - but applications for computing and getting work done?
Oh and don't forget about mobile devices - what amount of web apps do you prostulate people use there?
> SPA will be a cleaner architecture AND provide better experience to the user.
This. This is nonsense.
I will point out that I routinely use Reddit on mobile Safari and it is most certainly a web app. Things like Twitter, Facebook, etc. are also apps, and while you can download a native version, for privacy reasons I prefer their web app counterparts as they are less able to spy on what the rest of my phone is doing.
I will also again point out that Google Maps is a better experience than downloading OpenStreetMaps and setting it up yourself. Even something like Slack and Discord and Google Docs are perfectly good experiences on desktop browsers and require no installation whatsoever. You aren’t really addressing the zero installation point at all and that’s at the crux of why web apps are a thing and their main draw. You quoted my conclusion and called it nonsense without any basis since you provided a straw man argument and didn’t address the main point at all.
I have a personal toolkit of components I call "you might not need Javascript". Dropdowns, slideovers, toggles, modals, tabs, accordions, autocomplete, lightboxes, all with nice transition animations, and all in pure server-rendered HTML+CSS. A lot of it is just semantic elements being used appropriately. How many developers use <datalist> or <summary>/<details>? or :target and :focus-within? In my experience, not enough. How about, using <details> in combination with a CSS [open] selector? The palette gets rich in combination.
Overall, no, it's not hard, but it does require one to study. A couple of times a year, I'll re-read through the HTML LS and CSS specifications. There's a lot in there to build from, and almost all of it has broad evergreen browser support. For the leading-edge stuff, "caniuse.com" remains an essential reference. I currently have my eye on the <dialog> element, high expectations there, works well already in Chrome; Firefox just has a couple of bugs to shake out, and it's experimental (but incomplete) in Safari.
All that said: a light dusting of JS can still provide progressive enhancement, but we can limit this to minor cosmetic improvements & optimizations, for browser compatibility, and where dynamic ARIA markup is necessary.
The motivation as to why I prefer this style is another matter, and it's do with product strategy, and in particular, not crystallizing your architecture in the front end (where it is brittle) but in the backend (where it is much faster to pivot and/or rapidly prototype/spike/redevelop, by proximity to business logic and persistence schema). These are understandings that've been long (and sometimes painful) in the learning, but they're pretty much universal.
What is HTML LS?
The server side is being broken up into micro-services which makes all server a client/server roundtrip as well. Your DB was always running seperately making it a client/server roundtrip, etc.
Maybe I'm missing something, but what aspect of client/server is overly complex? And why is websocket (a client/server roundtrip mechanism) any easier?
The backend is likely all in the same building with a reliable and high-bandwith link between servers.
You can't compare that to a round-trip from the client to the backend.
Seriously, try phoenix liveview if you haven't already. These problems of dealing with stateful connections over questionable tranport and managing them are what the Erlang vm were designed for, and phoenix was designed by rails core team members.
If you instead have servers holding state, those servers become a lot less ephemeral.
In addition, HTTP has built in caching, which is a HUGE benefit for scale.
it works okay in small clusters on reliable low latency networks but it's not a replacement for an actual network mesh
there have been attempts to fix it but none have really found adoption. i think this is the most notable: https://lasp-lang.readme.io/docs
Wouldn't an intermediate layer such as e.g. Phoenix/LiveView (https://www.phoenixframework.org/) solve the problem?
I don't think so but if I'm wrong please correct me.
For example, imagine this simple scenario: you load the comment page for this post on HackerNews
HackerNews renders an HTTP response to your browser and your browser renders the page based on that HTTP payload.
You can be sitting here on the page for 10 minutes reading comments and even if HackerNews randomly went down 20 times for 10 seconds each time during those 10 minutes you would never know because your browser has everything it needed to render the page when it loaded. It's a done deal. The request was made and the response was served.
But now let's say you use LiveView and you made this page an actual Live View. Let's also say you load all of the comments at once to keep the example the same.
Since there's a persistent websocket connection open at all times, in all 20 cases of HackerNews going down you're going to get a visual warning that the websocket connection was dropped. You also don't want to disable this visual warning because on slower devices it might take 2 seconds to load the page which is much different than an uncontrolled disconnect.
IMO this user experience is pretty bad. Why should users be concerned with the server's state after it has everything it needed to handle the request it asked for (loading the comment page). If they try to post a comment and the server is down then the user should see that error at the time of them making the comment (such as a custom 502 error page if the back-end is down).
The grandfather's comment of "it wasn't designed this way by accident" is spot on. The stateless web is a happy web. Sure you can sprinkle in websockets for progressive enhancements like broadcasting new content somewhere (which is awesome when done well), but websockets shouldn't be the basis for everything. HTTP has decades of very careful thought and optimizations around its protocol.
It's due to Live View triggering a visual indicator that the websocket connection isn't available.
It's a combination of CSS and JS. You can turn this off but if you turn it off then you have no way to show an indicator when you do care (such as transitioning between pages or submitting a form).
I don't think LV at the client level can be coded in such a way that it can tell the difference between the server dropping and the user's connection dropping due to their internet being spotty.
To the rest...that's the point, it's just...CSS and JS. You can change it. You can show it or not, based on what the user is doing, or what view they have. Worst case, you can just hide it completely across everything, as you say, which is then no different from an SPA, except where functionality fails when you have no connectivity. Whether that leads to a better or worse user experience is dependent on the dev's handling of those failures.
You can restrict the visual warning to the first load only, it would be pretty distracting otherwise.
How would this work in practice while still showing it when you want it to be shown?
A feature that protects from ridiculously unlikely problem is no feature. It is over engineering
You could have incompetent engineers or you could have bad internet. The user could be on a train that is going in and out of tunnels. Or the user could be in a house using WiFi and the microwave could be causing interference.
I understand you needed an example, but a better comparison would be a heavy SPA with real-time interactivity. In that case, if your server is down your API calls will fail and you're going to have to handle the failure and let the user know something went wrong any way. If anything, Live View's built-in buffer and retry can actually provide a better user experience because the UI will automatically "catch up" when the client reconnects.
> You also don't want to disable this visual warning because on slower devices it might take 2 seconds to load the page which is much different than an uncontrolled disconnect.
Live View loads the entire initial page over HTTP, then mounts the Live View. Your slow device won't see any difference if the visual warning is disabled or not.
I have bad news for you about how often most web developers are using React.
It's because the creator of Live View himself said he's not going to be using "dead views" anymore for sites that don't need offline capabilities or are clear cases where LV would fall apart. He's advocating for everyone to use LV instead of regular views even to serve HTML for most sites.
I can see the appeal too, Turbolinks did a great thing a few years ago by improving page load speeds by turning regular HTTP requests into ajax calls behind the scenes and then swapping the <body> of the page, all without having to write a single line of code beyond importing the JS library and without having to change anything on the back-end.
With LV's live_redirect you can get a similar effect. It's a much different implementation (way more changes are needed on the back-end) but the end result is similar'ish.
And if you had a site that benefits from speedy page transitions (aka pretty much every site) and you happen to have a few cases where using LV makes sense such as wanting to do real time validations or show a stream of something then it would make sense to want to go all-in and use LV to serve every page so that you don't need to bring in both Turbolinks and LV as dependencies and still benefit by having fast page transitions across your site.
If you want to do requests in parallel, http 2 and 3 can do them just fine. If it’s sequential, you won’t see a huge difference between something on websockets and http/1.1.
The only thing you can save are a few header bytes. And it’s up to you how many headers you use
If you have authentication then cookies have to be going out also and any other information that has to be kept in order to provide state over a stateless connection. You still need to handshake on each connection (maybe http2/3 bring this down a bit), and you also don't have bidirectional flow, so if you need something resembling client side "real-time" functionality you now have to implement pooling, or server side events.
The biggest issue is basically how to load balance/scale and that becomes a problem only given a scale that 98% (scientific) of the websites online won't ever experience.
You can just use one connection (same as websockets)
> and you also don't have bidirectional flow
Sure you have. Each request and response has a body, and HTTP bodies are indefinite streams of data. You can do everything in them you like (including running your custom protocol). That's e.g. how gRPC streaming works.
And keeping one connection then we're in the same spot regarding the scaling issues as with web sockets right? Or does plain TCP/IP have better support for changing endpoints mid-flight, reconnections and etc?
Although if so I don't understand why there was even the need to come up with Websockets? I personally would love to see a simplification of technology all across OSes, web, etc, but probably there's too much out there to provide a reasonable migration path... It sometimes feels like those projects that you work on in the beginning and then you look at, some years down the line and say, this needs a refactoring with all I've learned in the meanwhile regarding what I'm doing and regarding my understanding of the domain space - but sometimes you can't do it, doing it might not go well or you'll end up in the same place afterwards.
And note that we're not talking about parallel request, those are not relevant to the "constant stream of small updates as the backend state updates" model described in the article, so let's not get side tracked, and talk about the actual stream of small fragment data in HTTP vs websockets.
Every HTTP connection (whether it's v1.1, v2, v3, or some even newer Google-flavoured "we came up with another one!" version) requires secure connection handshaking before any actual data may be sent. And if you're using any kind of session management, which you will be, now you're also spending time on cookie transmission before you get to payload transmission. And yes, there is the "Connection: Keep-Alive" header that can be set to allow more than one request per established connection, but any sane setup has that either on a small timeout (hundreds of ms at most), or on a low number of requests. You're still going to be establishing lots of connections, over and over, during the lifetime of the content at the client.
So, for many payloads a lengthy (compared to small payload transmission) HTTP connection negotiation has to happen. This is not the case for websockets, which by design are as if the "keep-alive" was set to "forever": they get initiated through a one-time HTTP call that contains an "upgrade this connection to a websocket" instruction header, after which we're done. This is now an open, persistent, bidirectional data socket.
Connection-wise, the amount of time required for HTTP vs. websocket is immense.
Secondly, let's not try to lend credence to an argument by hand waving the number of bytes in headers: the model discussed here concerns pushing small fragment updates to the client, in which headers constitute a significant part of the transmission. You can see this is modern sites that communicate small JSON fragments today where payloads like `{"error": false, "result": 12}` get dwarfed by the modern response headers, because yes "you control yoru headers" but you're not running your first server (in fact, you're almost certainly not running your own server at all, you paid a host): a response will contain not just the HTTP version and content type and length, but also the cache control value, the CSP and CORS headers, and almost certainly some extra stuff like the datetime, cookie expiration, x-frame-options, etc.
Using websockets makes an incredible difference in the model discussed in the article.
However, if you want to discuss the merits of the two completely different models, that's fair. Why would you even use the small-payloads push model? Who cares if the client has some state that "hopefully" mirrors the server? These are good questions, and unless you're writing a multi-user application with a web front-end, I genuinely don't see why the approach outlined in this article makes much sense.
Yeah, React kind of shat all over the web (and I say that as someone who quite likes React) because it was never designed with the purpose of generating web content. It was born as an interface library and never stopped being one. Its fault is being so damn easy to pick up which means that every site on the planet now uses React to build normal web pages and even do page routing, instead of having the devs write their code in React and then hitting the "and now generate that as static site with vanilla JS for the interactive bits" button. Because that button doesn't exist, instead it's esbuild (or previously, babel+webpack but thank god esbuild now exists so we don't have to use that anymore).
But that's a completely different discussion and really not related to how websockets are incredibly much better for the stateful server, thin client, many-small-updates push model.
The complexity benefit of having stateless HTTP transactions is vastly offset by the statefulness of TCP, TLS states, HTTP "sessions", cookies, browser storage & so on.
HTTP is hardly "designed" and plenty of its aspects were accidental.
Normally I’d agree with your take entirely and I don’t actually think it’s possible to avoid the complexity issues in non-BEAM languages...but Phoenix is absolutely doing it.
For all of the stateless goodness that HTTP is, in order to have user-specific content, you need to know who you're talking to. The hack to fix this was cookies. Instead of using a traditional stateful protocol (which could have kept track of the logged in user), we had a stateless protocol where the user basically had to present credentials with each request (also, HTTP-AUTH).
The second issue was so problematic that it made it into HTTP 1.1 two years later. Because a stateless protocol was used, if you wanted to download say, a document and a linked image in the document, in HTTP 1.0, you needed to make two requests. Back when each web page was predominantly text with few graphics, this was okay. When we moved to more graphical (albeit table-based) layouts, this was horrible. So, HTTP 1.1 ushered in Connection: keep-alive to allow for the same connection to be used for multiple requests. You still had to make individual requests, but at least you didn't have the overhead of setting up multiple TCP connections.
HTTP isn't fundamentally broken, but it is a kludge. The most painful parts of HTTP were also the most "stateless" aspects. I agree I don't understand why HTML over Websockets is a (big) thing. It doesn't make much sense to me either. Then again, I was also doing dynamic updates (with javascript pulling in HTML, not XML) before A List Apart (hello again) coined the term AJAX.
But let's not just say that HTTP is a perfect stateless protocol. It isn't. It was never "designed this way" -- it was patched up with quick fixes during the late 90's that we've been stuck with ever since. Websockets is the latest attempt to fix this problem. HTTP/2 and /3 don't really do anything to address maintaining session state, but focus on making individual request more efficient (roughly). That still leaves us with the problem of Cookies. Websockets have their own issues (scaling for one), but adopting the approach mentioned in the article doesn't mean that you have to completely abandon the rest of HTTP. But, if I could avoid having every website yelling at me about using cookies, I'd happily adopt a Websocket architecture, if that's what it took.
> That still leaves us with the problem of Cookies. ... But, if I could avoid having every website yelling at me about using cookies, I'd happily adopt a Websocket architecture, if that's what it took.
This is a red herring. If "the problem with cookies" is that websites yell at you about using cookies, that's a legal issue with the requirement to notify users they are being tracked. This "problem" of being yelled at didn't exist before the legal requirement.
Moving the transport to websockets (layered on top of HTTP, layered on top of TCP) doesn't get rid of the legal requirement for notification or opt-out. It just buys some time until the law catches up and says that the state management you're using with websockets is also subject to tracking notification and opt-out.
In fact, you'll probably find you'll need to store some tokens on the client, somehow, just to secure the client/server connection.
It’s not a red herring. And it’s not the website warnings that are the issue, it’s the tracking that they have to notify you about. If we didn’t have to allow cookies for basic interactions with websites (like a session cookie), then we wouldn’t need to allow tracking cookies at all. However, because we need to allow session cookies, other sites can then track us.
With a persistent connection/session, this wouldn’t be necessary.
But clearly, there are many other ways to tackle this problem. And the persistent socket connection also has issues. But after thinking about this, I think the company behind the current dominant browser may be a little conflicted about solving this issue. I’m hopeful about Firefox’s new approach.
Let's not go there again. It's the definition of worse than flash. And mostly for above described reasons.
Networking, Animation, Transitions, APIs, State of Data, State of Synchronization, Unpredictability and Unreliability of UI/UX States... all mixed together in a single socket.
Have fun debugging this blob of autogenerated code and transactions.
[1] www.gwtproject.org
Say you have a doctor that is associated to a group of patients and is allowed to update their patient's status. And if, while the doctor is filling something out for that patient, the patient changes doctors, then the current operation has to fail and the doctor needs to be notified of this change somehow.
So what do you do with the stale list of patients in the browser DOM?
Does the UI wait to see if the write fails, or does it increase reactivity by supporting a server push? And how do you tell the DOM to update the list of menu items so that that Patient X is removed from that list? What if half the doctor's patients have been removed? Does the UI wait for the doctor to try to make some change to all of them and one by one remove a patient or does it reload the full patient list? And if so, how often does the browser poll for the full the patient list -- during every operation involving a patient?
The server push reduces the amount of time wasted by the doctor filling out stuff for a patient that will be refused by the server anyway, and it avoids costly polling, reducing bandwidth. I get that it may not be worth it, it depends on how important reactivity and bandwidth are to the app, which is why videogame makers need to do it but people writing marketing pages don't.
Either way, you have a stateful connection as the set of patients available to that doctor is important connection state that can change at any time. So you are long past the idea of REST whenever you start building an interactive UI with data that must be synced from a remote server.
I am not a fan of the server push, but I feel that ship has sailed and people are adding in a lot more complexity for the sake of improved reactivity. My personal rule of thumb would be to always sacrifice reactivity for simplicity, but then this is why I don't build marketing pages.
Write the front end in something like Vue. When you catch an error from an axios call, update your data model by calling the server for updates.
When the data model updates, Vue will automagically update the DOM used by the browser.
Multiplayer online experiences are also growing to include more than gaming, such as betting and gambling and zoom and trading crypto.
Are websockets and or browsers improving to handle this server centric shift?
Then again, it wasn't designed this way to build applications with, either. It was designed to serve documents (the HT in HTTP).
All the problems with a stateful connection are solvable, but we have yet to figure out the right way to teach the discipline of doing it right. It is an exceptionally expensive proposition to solve, so most people shouldn't solve it as they will be picking up a huge bill.
The challenge is that HTTP is fundamentally broken for stateful experiences, and people are trying to build that into everything these days. They want shit to update across machines, and polling is exceptionally easy at great cost.
The HATEOAS client we use/build has an cache/event system that will emit to subscribers (via react hooks) when a resource is stale, and needs to be refreshed.
After putting this all in place, I realized that now I just need the server to emit those 'stale' events via Websockets, and suddenly my SPA + API is a real-time multi user system with nearly no changes required in either server or client.
This made me wonder, why aren't we just always pushing the entire resource state over websockets. Use websockets to let clients indicate which resources they are interested in, and let the server keep pushing state back to the client.
This solves a number of problems that plain HTTP has:
1. We can't really push/subscribe well. It's not very natural.
2. The HTTP cache kinda sucks, because we don't have programmatic access. Simple example: Doing a DELETE on a resource should expire the cache of the parent collection. There's a good HTTP header to indicate this, but no good way to tell the browser cache to delete the cache entry, so we end up building our own caching layer.
3. Compound requests suck because HTTP caches and clients don't understand them.
But we can still keep using REST fundamentals. Things have URIs, features are discoverable, relationships are expressed with links.
Something about this also feels terribly wrong though. Do we really want to reinvent parts of HTTP over websockets? HTTP is highly optimized, what kind of performance problems are we gonna run into?
The key reason is that a game lasts... minutes to hours (ignoring most MMOs), so the state machine makes sense and the risk of failure low. Web devs have to use a database which has state for years.
I'm not sure we should be taking advice about how to load information on mobile devices with spotty connections from the gaming industry.
Bi-directional gives you more power, but you also have to think harder about the protocol and application state once the server is also pushing events to the client unprompted.
I ask because I rolled out a REST/HATEOAS api for a multi-org project and got zero uptake on the HATEOAS aspect of responses. The devs hardcoded URL resource links and totally ignored any links in responses . . .
After that experience, I am always interested to hear about more successful REST/HATEOAS projects.
Because this won't respect HTTP caching and you'll end up sending way more data than you need. Best instead to just send up URI and `GET/DELETE` verbs to indicate intent. I wrote about this: https://github.com/krainboltgreene/pubway.js#pubway
Actually every HTTP benefit is a reason why you wouldn't want to send resources over websocket. What if the websocket connection is slower than changes to the resource? Instead of simply telling the client what to fetch, you just gave them stale data.
This whole saga feels like one of those things where "I've heard this story before, I think I'll sit this one out and hope it's not too long" situations.
We all prioritize what tech we're going to spend our time on, but I think there's something qualitatively different between, "that could be interesting but I don't care", "That'll never work!", and, "I understand this enough to know it does not meet my criteria for job satisfaction, I'm out. Call me when the Trough of Disillusionment hits."
So we come back around to Rails, 15 years on from its launch…
Is it really that bad?
With Turbolinks 5 (and its successor Hotwire Turbo Drive) you can do nothing except add 1 line of JS to your application and reap massive wins in terms of page transition speeds. And this all works without changing 1 line of code on the back-end and it works with all tech stacks.
IMO it's the biggest bang for your buck that you can do to make a website feel good with the least amount of effort.
And now with Turbo Drive you can do the same with form submissions without needing any back-end code and it also works with every tech stack with no special libraries or work to be done server side.
Then Turbo Frames builds upon that to partially update a segment of your page and Turbo Streams introduces websockets for when you want to broadcast stuff or do real-time updates.
It's a great system. Use HTTP for everything except for when websockets makes sense.
I decided to look it up and Intercooler is still supported, but there is a successor now called htmx. I have not shipped anything with either, but I think they are both technologies worth taking a look at. It also looks like htmx is working on support for SSE and websockets.
Modern web development is a big reason, I left my job at the end of 2019 and went back to school.
I'm tempted to try something with Javalin(a fork of Spark), j2html + htmx, and Tachyons css now. I think together those would make a beautiful stack.
The missing piece here is that you still need those platform-agnostic APIs and an absolutely clean separation of your backend with all the business logic and your presentation layer. But now your presentation layer mostly lives on your servers and is doing SSR and pushing data over websockets. It takes real discipline to not commingle the two because you’re right there you could just peak at the database a little smack.
The other hard thing about this approach is that now your frontend presentation servers have to keep their own state since they are the clients now which means they need shared storage of some fashion and likely even shared persistent storage if you really want to take advantage.
> Want a hyper-responsive datalist typeahead that is perfectly synced with the database? On every keystroke, send a query down the WebSocket and get back precisely the changed set of option tags, nothing more, nothing less.
> How about client-side validations? Easy. On every input change, round up the form values and send ’em down the WebSocket. Let your server framework validate and send back changes to the HTML of the form, including any errors that need to be rendered. No need for JSON or complicated error objects.
If you're not Google-scale this is just a good way to DDOS yourself, no?
Should you run an unindexed query on every keystroke of a text input of a form for every user? No.
But if you're not a junior dev, you'll know not to do those things and you'll know to debounce the text inputs, plus maybe only query the DB is the input text is longer than n characters for example. And you'll use a GIN index at first in your Postgres DB for fast text search.
It might all sound a bit insane but I believe that everyone should give Phoenix/Liveview a try. That stack can scale and it works amazingly well even on a cheap Heroku dyno.
Also, every background tab in the user's browser now consumes server resources although idle.
You do not need this for your 1 to 5 stars rating thing. Most web pages do not need this. They're rather banal. We have huge overkill in web technologies now. We have WebGL, which is little used, and WebAsm, which is little used, for when you really, really want to have full interactivity in the browser. There are a few nice applications using those, but only Google Earth is mainstream.
Nonetheless the idea is not bad. I just feel that HTTP/2 with its multiplexing is a nice middle ground and it comes for absolutely free.
After reading, I almost laughed out loud that the author apparently felt the need to relitigate the rise of SPAs, throw in a shoutout to SEO, dunk on Java developers, tug on our heartstrings thinking of the kids in Africa with iPhone 4s crying while Safari parses 100kB of JSON (in 2005, this would have been a story about grandmas surfing the web on IE5), and, as a special treat, throw out a recommendation for Ruby on Rails in 2021.
A List Apart appears to have pivoted to a sort of digital Renaissance fair, but for the year 2007.
Simply put a client-side SPA is much cheaper to run than almost anything else because you're pushing as much computation as possible onto the client where you don't pay for it. You just need a good static host with a CDN like Netlify, Firebase, etc which is nearly free (for small sites, completely free).
Developers who have tried this are not keen to go back to paying for servers to do this computation. Big companies can and will pay for this cost to improve user experience (see: Next.js popularity) but I just think a lot of solo hackers will shy away from it and that will affect momentum.
This isn't really a critique of the technique one way or the other, just a prediction of how far the trend can go.
I also feel there is something to ownership of your tools and platform, and an application strung across several SaaS and a toolset that requires phoning out to different SaaS in order to work is anything but yours.
Maybe in an ideal world where responses are 1ms this would work, but imagine real world, real-time validating user input (ie email) and the user needs to wait before they find out their input is valid.
What you may want to do is to submit changes in case you want to see if an email has been taken already, for instance, to display additional information while the user is writing and this no matter what technology is used would always require a round-trip? No saying that you need that, but it's up to you when implementing deciding on what is the UI/UX you need and take it from there, it's not like it forces you to do that.
(sometimes articles take it a bit too far and end up showing anti-patterns, mixing problems it solves and problems it creates when misused)
Don't get me wrong - we use Blazor server-side for several critical webapps now, but I would be the last person to suggest you use it (or anything like it) for all web applications. For instance, a Netflix-scale operation + Blazor server-side would almost certainly be a really bad mix.
For use cases where the maximum number of possible users is bounded by ~10-100k and within a 100ms latency domain, I think a very strong argument can be made for such technologies. The productivity uplift is pretty incredible when you can directly use your existing domain services to drive UI.
We also like the security implications of these sorts of technologies. The incremental disclosure of the client app is an excellent selling point for many of our customers.
Umm... not it hasn't? Separating the backend from the frontend allows for multiple frontends on multiple types of delivery. Outside of edge workers, how is HTML-over-websockets supposed to compete with a good ol CDN (jamstack) on TTR?
That said, there is more operational complexity (though given my site is relatively small, not much yet). State that lives with the server dies with the server, so deployments can take longer if client sessions live for a while.
But instead reminds me of a friend who went full gRPC learning, so far as to start contributing to the newer standard definitions of RPC, he used to turn around and say that all API's SHOULD be replaced with gRPC and argue as I would about how its not always needed etc, he knew enough to counter with good reasons instead or the reasons why it should be.. And sometimes thats the biggest hurdle, its not that it can do everything better, even easier. It's just the commitment devs/companies whoever will have to do it..
I'm guilty of not using the right tool for the job sometimes.. But I'm struggling to think of any protocol or implementation that has been fully replaced without huge intervention..
Even TSL/SSL implentations was (rightly) pushed heavily by browsers, no ssl, no sensitive data transfer. But on its own, I don't see any replacements today wiping out the old.
I'm no fan of latest fads.. You show me a good standards doc thats got strict implementations, I'm on board! You show me a 'recommendation' based on some amazing points and valid critique, I'm not so enthusiastic..
And by local, I mean running in the same page.
Turbo is actually a rather simple way of achieving this with rails https://turbo.hotwire.dev/handbook/streams
Which is something that would be very clunky for stimulus
Not sure about React, but with Angular and tools like gRPC you don't see a single byte of JSON and the code that interacts with the APIs is automatically generated at compile-time and is type safe (with intellisense too!)
It is pretty effortless these days IME. You can't make a mistake with the coding side of things calling an API as the actual mechanics is seamlessly hidden away - mistakes can only be made in the "business" side of things (and this HTML-approach won't save you there...)
If you are using an "in house" framework (or just something ancient/niche like rails) then perhaps you don't benefit as much from modern bug-resistant frameworks and tooling I guess... In which case you're probably going to make more gains just by using a mainstream framework rather than trying something new/unusual/untested(?) like this.
Good luck anyway!
* not everyone has “two codebases” many people have one codebase and two different application distributions, one for server and one for client.
* you will still have “two codebases” you have just removed the rendering logic from you client codebase.
* What is the distribution of latencies of rendering json into html on the client? Are there numbers to suggest this takes more than 100ms for many people? If not the proposed efficiency advantages are being overstated.
If the article were suggesting removing client side logic completely these arguments would be stronger.
There is a lot of talk at the start of the article how web development should be easier, how it should be less about the underlying architecture and that we shouldn't spend time on implementation details like state that SPA suddenly has to manage. On the other hand, I felt the rest of the article seemed pretty implementation focused, talking about details of how sockets could be used - I do have decent web dev experience but grasping the practicality of the concept based on what was described was beyond me.
That said, I love the first part that is talking about simplifying the web app development, to make it similarly simple as it was 10 years ago, but I can hardly imagine that solution to that is replacing one piece of underlying technology with another piece. I imagine that instead, we need a higher level of abstraction that will hide all the implementation details that we don't care about and capture that what is common through time, so we don't have to care if we are using websockets or http or smth else underneath - instead we say how we want data to behave, and it gets done. Dealing with HTTP or sockets should be reserved for advanced use cases, not typical web apps. I am working on a concept of this, open-source web dev DSL, still in alpha though: https://wasp-lang.dev , and while I can't be sure that it is the exact solution, I imagine something similar in the future, a DSL. At the end, frameworks are a step in that direction (embedded DSLs), it is just that they are not standalone languages and are therefore again pretty coupled with the language and architecture they are modeling.
In the SPA model yes, they will have to wait a long time to download the initial JS payload, but that can be cached. Thereafter, if UI updates don't need to talk to the server, they can be computed client-side and the UI can update instantaneously. But in the suggested model, a much larger class of client-side actions would trigger (blocking?) network requests.
HTML over the wire is often a useful approach; it doesn't follow that "the future" is there.
Apollo’s GraphQL with built in pub sub and JSON over websockets solves most of the issues brought up here with little work.
This seems like unnecessary overhead for any startup software companies, costing both extra time and money while overloading your server right out the gates.
Instead of expecting 100 requests per min from say like 20 users, I can now expect 1000s+ to my server because I'm sending every keystroke to the server ???
That means I'll need a super expensive instance to handle all those users and this is in the extremely conservative use-case of 20 users at once.
No thanks.
Doesn't this go away with HTTP/3 and wherever QUIC is used?
It is almost 30 years since the web came into existence and somehow people harbour this illusion that those who control the standards or are responsible for implementing them are genuinely committed to delivering technology which will help the competition (ie the rest of the world) bypasses their app stores.
It makes one wonder whether those who keep blabbing on about these wonderful new technologies are shills or just stupid.
Do they believe that a company that patented round corners and tie batteries to a particular phone to prevent it from working on another of the same make to be committed to technologies that bypasses their app stores?
Do they think companies which ban apps from their stores as a consequence of their political partisanship will create an easy and effective bypass for them?
I see someone mention Blazor below? Why not Silverlight? Oh it is proprietary.
Well there are only two players in the browser market now, Apple and Google, and somehow between the two of them they can't come up with an NPAPI replacement although one of them makes the other happy to the tune of $12bn a year.
It is 10 years since these companies conspired to deprecate Flash on the grounds that it was buggy and performance was poor. 10 years on phones and computers are an order of magnitude more powerful, yet through some mysterious means the replacement technologies are still buggy, underperformant, need more refinement yada yada yada.
Why do otherwise intelligent people continue to put with this fraud, or is more a matter of Stockholm Syndrome?
If you combine the approach of WebSockets + dom patching with postgres pg_notify you get some real magic :) In the new IHP framework we provide something like that. We call it Auto Refresh: https://ihp.digitallyinduced.com/Guide/auto-refresh.html
Auto Refresh is activated with a function call inside a controller action, like this:
action ShowProjectAction { projectId } = autoRefresh do
project <- fetch projectId
render ShowView { .. }
Once activated IHP will automatically track all tables your action is using e.g. in `SELECT * FROM ...` queries. Once the action sends a response IHP will start watching for any kind of `INSERT`, `UPDATE` or `DELETE` statement to all the tables used by your action using pg_notify.
When the page is rendered a small JavaScript function will connect back to the IHP server using a WebSocket connection.
Whenever an `INSERT`, `UPDATE` or `DELETE` happens to the tables used by your action IHP will rerun your action on the server-side. When the generated HTML looks different than the HTML generated on the initial page load it will send the new HTML to the browser using the WebSocket connection. The JavaScript listening on the WebSocket will use the new HTML to update the current page. It uses morphdom to only touch the parts of your current DOM that have changed.
Here's how this looks in the real world: https://twitter.com/digitallyinduce/status/13207288333943808... <- This view has no app-specific Javascript. The auto refresh is enabled using a single function call and the framework takes care of all the technical details.
Some use cases where this really shines:
- interactive dashboards (e.g. showing the current status of job queue)
- simple user chat systems (you can send messages using normal form submission, as long as auto refresh is enabled this works like a SPA)
- building a twitter clone, the feed will automatically add new content as it's added to the database
We're using this approach in production since last year and it saves an incredible amount of development time.
Use components as if you were doing front end on the server. No more crappy server templating engines with poor composition and no interactivity. You go straight from the DB to the component. Then only the absolute minimum JS needed for interactivity is sent to the client.
This is not so far away btw.
I made this work on Svelte although hydration is clunky and it was a bit convoluted.
Marko has been doing this since 2017 I believe and is running on Ebay.
AFAIK Imba 2 will have partial hydration too.
No kidding though, Poe's Law applies to this article. This shit has gotten out of hand. A List Apart used to be a serious publication; ideas like progressive enhancement and responsive CSS came right out of that publication.
The fact that many folks think this is a good idea paints a bleak picture of the future of web development.
The web worked well before (circa 2010, jQuery, Ajax, etc.). It was just too complicated to make very dynamic pages.
We don't need more inventions. We need to use the inventions we have correctly.
Instead we got Angular, et al.
The ability to connect DOM elements to state management is great. This was another piece we needed. Unfortunately, this piece was introduced along side a pattern and framework. The former lent credibility to the latter two. This same thing happened when rails came out; rails introduced the ORM to the masses, and offered MVC along side it. The vertically integrated synergies created by these parts allowed rails to soar. In that case, it was a good thing; Rails changed the web for the better, by a long shot.
The rabbit hole that reactive took front-end web developers down over the past decade has been more than destructive, because it happened just in time for the largest growth in new web developers ever. Primarily, this is the reason it was able to happen (the overall reduction in aggregate experience).
These frameworks have taught us a lot of really great individual things, but as whole they have ruined many parts of the web, and they've destroyed the discipline of developers. Now a basic CRUD application is dynamic enough, for many developers, to warrant the use of such frameworks.
You can say what you want about how tremendous this reactive framework or that reactive framework is, but you cannot deny the fact that 75% of websites have hundreds of percent more bugs, and nearly half the web renders a blank page for multiple seconds before anything shows up to the end-user. This is worse than the 1990s. Honest to God, I give up on more than 1/3 of e-commerce transactions these days, because some form of UI fuckery breaks my will. When I open a link and the whole page is white, despite seeing that everything has loaded, I literally close the tab before even having to see the monstrosity that they've created. What's worse is that many developers will create a reactive website that literally does not react; it just changes pages and re-reacts every time a new page loads. SMH...
</rant>
The trends here are that computation can be remote and that real time, responsive interactive UI can be remote as well thus removing a lot of the need for a lot of complex local on device interactions. The only thing that goes over the wire to the server is low level input events; the thing that comes back is intended for a software renderer written in WASM and Web GL. You should not have to build your own renderer. Gaming companies ship existing games to things like Stadia or Nvidia Geforce, etc.
Not saying that this is the future. But we seem stuck in a lot of abstractions and constraints imposed on us in the late nineties when Brendan Eich et al. gobbled together a scripting language in a hurry and HTTP 1.1 was frozen in time until SPDY became a thing only a few years ago. Most of the past 20 years is just increasingly more sophisticated hacks to work around andundo all of that. We are now at the point where you can play an FPS game running on a remote server or boot a virtual machine with an actual operating system in your browser. So, why are we still doing UX with our hands tied behind our backs? Is that really the state of the art? IMHO it's not even close to that. It wasn't the state of the art 20 years go. All it ever was was convenient for developers. And even that is debatable.
We had graphical tools for clicking together UI in the nineties (Delphi, Visual Basic, JBuilder, etc.). That stuff worked great. OK, it resulted in a lot of ugly and unusable applications. But then most SAAS browser applications that replaced them aren't a whole lot better or less painful to deal with.
Yesterday, I was in a design meeting where we looked at our latest Figma designs. Figma is a tool for clicking together UI just so you can create something that looks like an app that you then give to a developer so they can transform it into something that actually works. That process is completely manual (most Figma users don't generate code from it). UI frameworks and tools are terrible. All this manual work that you need to do to get from a slick click demo that runs in a browser to an application that looks exactly the same and works .... in the same browser. Am I the only one who thinks that is backwards?
So moving some of that back into our servers is a baby step on the way to better tooling. But we have a long way to go.