How we got to LiveView
fly.io
fly.io
It can be daunting to jump into such a strange world as a LiveView environment may look (Elixir syntax, OTP terminology, etc) but honestly once you dig in deeper, everything just makes sense. LiveView (and HEEx) continue to be very simple to understand abstractions on top of the rock solid OTP platform. It's a joy to build real time applications using it, and I very much appreciate the "developer experience" focus both Chris & Jose have for us Elixir devs!
I'm excited for the launch of Phoenix 1.6 and HEEx is shaping up to be a complete replacement for your traditional SPA + Backend API, and using one consistent language for your full stack really has very freeing & powerful benefits, especially for small teams!
In any case, I'd love to chat with you on my podcast on how you built and deploy Glimesh if you're interested. It's at https://runninginproduction.com/, there's a become a guest button in the nav bar on the top right if you wanted to schedule a call to be on the show.
Thanks for the offer to be on the podcast, I'll check it out!
[1] https://htmx.org
I've read the blog post about getting 2 million concurrent connections on a single server with minimal tuning. That's pretty mind blowing. How does that compare will real-world scalability with LV? Like, can I actually have 2 million people looking at a LV simultaneously? I think the worry I hear most is whether or not LV scales. I've personally never run into any problems at all, but the projects I've built with LV haven't been the highest-traffic apps.
The other thing to consider load-wise is the very state that you can now keep in memory allows you to reduce system load. Instead of fetching and authenticating the current user for every interaction, you do that one time for the lifetime of the visit. Database load is drastically reduced and you don't spend CPU cycles doing token verification. So while there's a cost to holding state, in general this will allow you to do much less work than a stateless application.
One question I keep thinking about. What if I want to still use React on the frontend, does it still make sense to use Phoenix for the backend then or am I throwing all benefits over board then?
Edit: I realize you could just use regular web sockets with Phoenix to do what Socket.IO does. But my confusion comes from actually getting it all to play nice with a LiveView front end, as well as a native app front end.
The Getting Started guide on Phoenix has a lot of useful material even for your use case. I might be missing something crucial though :)
The channel docs give a solid overview of the server side, and we have a listing of third-party channels clients for most platforms here:
https://hexdocs.pm/phoenix/1.6.0-rc.1/channels.html#client-l...
So to answer your question directly, you could either add your JSON or GraphQL API directly in the same router that serves your LiveViews, or you could create a router specific to the API, or you could introduce a completely separate Phoenix endpoint and router that starts a 2nd web server. Both would boot as part of the app serving different ports. Phoenix remains a great choice for native clients if we're talking JSON/GraphQL, but because we also have native channels clients in objc/swift. Hope that helps!
Of course, LiveView isn't MVC in the classic sense, but it still uses plugs. Its goal is to simplify the SPA which I'd say it does a particular good job of.
[Edited to fix part of a sentence I deleted by accident]
And having to go through a few files to understand how a request is handled is not out of the ordinary in an app, especially if it's grown over the years?
For example:. What exactly is the distinction between an endpoint and a router?
You can have multiple routers. I just built a thing where foo.com uses one router for the main site and *.foo.com is something else.
Ignore the "View" modules. Put nothing in them. You can also eschew the "Context" modules for your database stuff, and just use Ecto functions directly, though, if you do get into the habit of using Context modules, it will make your life easier to have a unified point of view that abstracts away, e.g. caching, or, if you decide to go full CQRS/event-driven, if you need to send a copy of your data to a datalake, etc, etc.
Actually LiveView tends to have less module-indirection than "deadview"s.
If you prefer to have simple apis, you can just drop phoenix altogether and just stick to Plug. This will give you a ruby/sinatra-style experience that may be more your style, and, IMO, a good way to learn elixir. I did this for a long time before finally giving in and learning phoenix. I turned out OK.
1. It is much easier to trace things through the entire Phoenix stack than it is in Rails. It is also much easier to add things to the Phoenix stack using plugs.
2. Elixir is concurrent, whereas Ruby is not, so when performing long-running processes, you can just do them in Elixir/Phoenix without having to rely on workarounds like Sidekiq, Resque, RabbitMQ, etc.
3. Writing multi-threaded applications is much easier in Elixir than many (if not all) OO languages.
4. Pattern-matching for variables and functions, and binary pattern-matching for parsing text.
5. Mix.
6. BEAM, OTP, and supervision trees.
"When we want to have more control, i.e. persistence on disk, a workers pool, retries, among other things, there are several popular solutions:
Oban: a robust PostgreSQL-based queue backend process system. It has a UI in its pro version Exq: a Redis-based Resque and Sidekiq compliant processing library. It has an open source UI exq_ui verk: same as Exq. Is is compatible with Resque and Sidekiq job definitions"
https://sipsandbits.com/2020/08/07/do-we-need-background-job...
So basically sounds to me like Elixir is doing pretty much the same thing as Ruby. We usually do want more control and disk persistence, so...
edit: > BEAM, OTP, concurrency, this whole thing is solved with the current fashion of devops teams and kubernetes.
This is a hilarious statement which I hope is satire
I do agree with you that's not a huge win compared with Rails, but it is nice to have. I think you'd have to look more at something like "lots of concurrent, long-lived connections" for the real wins over Rails for the BEAM ecosystem. I mean, you can do that in Ruby if you want to, but it's going to be cleaner and simpler with Elixir/Erlang.
> Queue Control — Queues can be started, stopped, paused, resumed and scaled independently at runtime locally or across all running nodes (even in environments like Heroku, without distributed Erlang).
The advantages of BEAM and OTP are that I can spawn new concurrent processes throughout a cluster and then use the actor model to send direct messages to those processes from any other processes, regardless of where the sender and receiver happen to be. I can also easily configure Behaviours so they automatically run on either a single node or every node in the cluster, depending on what they are doing.
In both cases, if a node goes down, Elixir will automatically restart any affected processes on another node. When a process has an issue, I can surgically handle that issue at the individual process level. Kubernetes doesn't handle problems that granularly.
One advantage of Kubernetes is autoscaling. OTP and BEAM don't do that.
By using libcluster, you can combine Kubernetes and BEAM/OTP and get the best of both worlds.
Can you talk a little about the current story with deployments and managing state across/during those?
Obviously Erlang/OTP supports hot upgrades, but those are hard to design correctly and not supported by container/VM environments like Heroku and (I presume?) Fly.io.
With this and a DB to store permanent data, an LV app can be deployed just like any other standard container app without user impact.
So you opt-in to this complexity when you have a clear requirement for it. They other key point in this, is in either hot upgrade or cold deploy cases, you still have to design your stateful system to recover from complete restart. Servers catch on fire and things go bad sometimes, so the cold deploy approach of rebuilding the state on restart is not only completely viable by itself, but you're doing it anyway even with hot upgrades. Hope that helps!
We intentionally give you super user access to your DBs so it's easy to migrate or spin up your own replicas. Our bet is that _most_ people won't outgrow our Postgres but having an escape hatch is still very comforting.
We previously worked on Compose.com. We, at least, understand how to manage huge databases when the time comes.
I know there is this work in progress (https://w3c.github.io/webtransport/) and websockets are probably fine for a long time but sooner or later (unless there is an update to websockets) it will probably be faster to just do normal http requests and listen on server sent events.
What are your thoughts for Liveview for the future? Will it forever stay on websockets or would you be open to change the underlying technology if / when new stuff becomes available?
It is such an awesome technology.
I got a couple of things:
I seem to remember in the original announcement presentation there was a demo of SVG being updated inside a page 60 times per second. All from server. Did this actually become feasible? I’m thinking graphs and maps with live data. I might not need as smooth animation there. Though that could make for nice dashboards.
Other bit that interests me is web apps for long running tasks. What’s the story in Phoenix and Elixir land for handling external shell processes from web requests? I’m trying to do lightweight job control without investing into separate systems for processing pipelines, CI/CD and the like.
Thank you for Live View. It continues to be an inspiration even though I haven’t yet had a chance to dive into server side of it.
That said, you are right that SVG charts/maps are a surprisingly fantastic fit for LiveView. You could actually render a fully interactive and dynamically updating chart by only sending SVGs – and it will probably send less data than hydrating the same client-side chart with JSON! Check out the Context Chart lib for some examples. They have some links running LiveView examples if you scroll down: https://contex-charts.org
> What’s the story in Phoenix and Elixir land for handling external shell processes from web requests
Great story here thru erlang ports. I'm a big fan of the Porcelain library which wraps ports with some nice features on top: https://github.com/alco/porcelain
From what I understand, the issue is that every event that happens on the client (say, a click) has to make a roundtrip to the server before the UI can be updated. If latency is high, this can make for a poor user experience, the argument goes.
As the creator of LiveView, what’s your take on this? Is it a real and difficult-to-solve issue, or do people just not see "the LiveView way" of solving it?
I think LiveView looks amazing, but this possible issue (in addition to chronic lack of time) has made me a little unsure of whether it’s ready to use for a real project.
Thanks for creating Phoenix, btw!
> If latency is high, this can make for a poor user experience, the argument goes.
Deploy auto-scaling servers closer to your users: Use fly.io (or any other competent edge platform, really).
For example, if I have a flag to show/hide a modal to confirm a resource delete, the show/hide flag would live in AlpineJS, while the resource I was deleting would live in the state of my LiveView.
This way, there are no round trips to the server over websocket to toggle the modal. Hopefully that example makes sense :).
I just found out that Livewire was inspired by LiveView.
This means moving logic to client side JS becomes an optimization, rather than a requirement. You can naively build LiveView and send stuff you shouldn't to the server, then optimize that away by writing javascript as your app matures.
What I've found is that I don't get to the optimize with JS step very often. But I know it's there if I need it.
One of YouTube’s most pivotal moments was when they saw their latency skyrocketed. They couldn’t figure out why.
Until someone realized it was because their users, for the first time, were world wide. The Brazilians were causing their latency charts to go from a nice <300ms average to >1.5s average. Yet obviously that was a great thing, because of Brazilians want your product so badly they’re willing to wait 1.5s every click, you’re probably on to something.
Mark my words: if elixir takes off, someday someone is going to write the equivalent of how gamedevs solve this problem: client side logic to extrapolate instantaneous changes + server side rollback if the client gets out of sync.
Or they won’t, and everyone will just assume 50ms is all you need. :)
This isn't the takeaway at all. The takeaway is we can match or beat SPAs that necessarily have to talk to the server anyway, which covers a massive class of applications. You'd deploy your SPA driven app close to users for the same reason you'd deploy your LiveView application, or your assets – reducing the speed of light distance provides better UX. It's just that most platforms outside of Elixir have no distribution story, so being close to users involves way more operation and code level concerns and becomes a non-starter. Deploying LiveView close to users is like deploying your game server closes to users – we have real, actual running code for that user so we can do all kinds of interesting things being near to them.
The way we write applications lends itself to being close to users.
Then why do you start running forward instantly when you press “W” in counterstrike or quake? Why not just deploy servers closer to users?
Gamedev and webdev are more closely related than they seem. Now that webdev is getting closer, it might be good to take advantage of gamedev’s prior art in this domain.
There’s a reason us gamedevs go through the trouble. That pesky speed of light isn’t instant. Pressing “w” (or tapping a button) isn’t instant either, but it may as well be.
You do both? Game client handles movements and writes game state changes to a server, which should be close to the user to reduce the possibility for invalid state behaviors? You really haven't seen online games that deploy servers all over the world to reduce latency for their users? What?
Both web apps and games do optimistic server writes. Both web apps and games have to accommodate a failed write. Both web apps and games handle local state and remote state differently.
1. SPA with asynchronous server communication. A button switches to a spinner the moment you click it, and spins until the update is safe at the server. Error messages can show up near the button, or in a toast.
2. LiveView where updates go via the server. The button shows no change (after recovering from submit "bounce" animation) until a response from the server has come back to you. To do anything better, you need to write it yourself, and now you're back in SPA world again.
There's a reason textarea input isn't sent to a server with the server updating the visible contents! Same thing applies to all aspects of the UX.
EDIT: https://dockyard.com/blogs/optimizing-user-experience-with-l... talks about this. That'll handle things like buttons being disabled while a request is in flight, but it won't e.g. immediately add new TODO items to the classic TODO list example.
Imagine how painful typing would be if you had to wait after each keypress till the server acknowledged it. Everyone’s had the experience of being SSH’ed into a mostly-frozen server; good luck typing on a phone keyboard instead of a real keyboard without typo’ing your buffered keys.
The point is, there are many application-specific areas that client side prediction is necessary. Taking a hardline stance of “just deploy closer servers” will only handicap elixir in the long run.
Why not tackle the problem head-on? Unreal Engine might be worth studying here: https://docs.unrealengine.com/udk/Three/NetworkingOverview.h...
One could imagine a “client eval” code block in elixir which only executes on the client, and which contains all the prediction logic.
While we can draw parallels to game servers being near users, I don't think it makes sense for us to argue that LiveView should take the same architecture as an FPS :)
Anyway, you rock. :)
The trick was to use HN search: "youtube latency" and select Comments. First result was a comment on https://www.forrestthewoods.com/blog/my_favorite_paradox/ which links the story in the "Bonus!" section.
most games have the benefit that they're modeling the mechanics of physical objects moving around in the world and are having their users express their intentions through spatial movement. the first gives a pretty healthy prior in terms of modeling movement when data drops out and the latter can be fairly repetitive and thereby learnable and predictable.
whether or not user interaction behaviors can be learned within the context of driving web applications seems a little less clear, to me at least. it does seem like there are a lot more degrees of freedom.
server controlled client side rollback, you mean?
First off, it's important to call out how LiveView's docs recommend folks keep interactions purely client side for purely client side interactions: https://hexdocs.pm/phoenix_live_view/Phoenix.LiveView.html#m...
> There are also use cases which are a bad fit for LiveView:
> Animations - animations, menus, and general UI events that do not need the server in the first place are a bad fit for LiveView. Those can be achieved without LiveView in multiple ways, such as with CSS and CSS transitions, using LiveView hooks, or even integrating with UI toolkits designed for this purpose, such as Bootstrap, Alpine.JS, and similar
Second, it's important to call out how LiveView will beat client-side apps that necessarily needs to talk to the server to perform writes or reads because we already have the connection established and there's less overhead on the other side since we don't need to fetch the world, and we send less data as the result of the interaction. If you click "post tweet", wether it's LiveView or React, you're talking to the server so there's no more or less suitability there compared to an SPA.
I had a big writeup about these points on the DockYard blog for those interested in this kind of thing along with LiveViews optimistic UI features:
https://dockyard.com/blog/2020/12/21/optimizing-user-experie...
Between things like phx-disable-with and phx-*-loading, and the ability to add any client-side logic using JS, there doesn’t really seem to be any limitations compared to a more traditional SPA using (for example) React and a JSON API.
I hope I haven’t added to the confusion about this by bringing it up, I was just very curious to hear your thoughts on it.
I’ll grant you that that isn’t often the case, and recovering from inconsistencies is pretty painful, but I can see how people would go for that.
I kind of like the idea I can just build all my code in one place instead of completely separate front and back-end though.
Don't modern browsers already share a TCP connection for multiple queries implicitly?
Example:
A full fresh HTTP connect from client to first byte take ~400ms(I'm in the US the server is in Europe). This includes: resolve dns, open tcp connection, ssl handshake etc...
But if the connection is already establish, it only takes ~200ms to first byte.
If I deployed the server in the same region, say US customer <-> US server, this came down to 20ms...
That means, it's good enough.
Not super ideal but It's a trade-off we're willing to make.
A large majority of businesses out there start off targeting one region/area/country. The latency from LiveView in this scenario is imperceptible (it's microseconds). If these businesses are so lucky as to expand internationally, they are going to want to deploy instances of their apps closer to their users regardless of wether or not they are using LiveView.
LiveView could be a huge help to these startups. The development speed to get a concurrent, SPA-style app up and running is unparalleled, and it scales really well. My guess would that be that people who are worried about the latency here (which is going to exist from any SPA anyway) are the ones who are developing personal pages, blogs, educational material, etc. that they are hoping the world is going to see out of the gates. In this case, LiveView is not the answer!!! And as I've stated elsewhere 'round here, LiveView does not claim to be "one-size-fits-all". If latency really IS that big of a concern, LiveView is not the right choice for your app. But there really is a huge set of businesses that could really benefit from using it, either because they are a start-up focused on a single area/region/country, or they are already making tons of money can easily afford to "just deploy closer to their users" and could benefit from LiveView's (and Phoenix's) extreme simplicity.
It’s not designed to run the NY Times. But it is a super useful tool that will benefit a ton of apps out there.
Edit: Just saw that this was already pointed out. Apologies, didn’t mean to pile on.
Agree it's super neat framework and hope a client side implementation can be written to "stage" the change.
Here’s the link: https://dockyard.com/blog/2020/12/21/optimizing-user-experie...
http://blog.pthompson.org/liveview-tailwind-css-alpine-js-mo...
In this pixel paint demo, the state and reaction of "changing the pen color" can happens locally: https://github.com/beenotung/live-paint/blob/dd3b370/server/...
Do you think it is possible for newcomers to pick up Phoenix and at the same time learn how frameworks work? For example, FastAPI [0] is a widely extensive Python framework yet its documentation essentially is just a long tutorial which explains basic web-development concepts while teaching its core. OTOH, almost all Phoenix learning material I have come across so far are targeted to experienced devs (especially Rails).
I really love that Phoenix is opinionated but I feel like there should be some ways to learn about the rationale behind those choices, without having 10 years of development intuition.
http://www.petecorey.com/blog/2019/05/20/minimum-viable-phoe...
We've been using LiveView to build a new app at Precision Nutrition and are largely quite happy with it so far.
One concern we keep coming back to is that of the need for constant connectivity in order for the app to work. I'll throw up the disclaimer here that I've not spike on how to handle network disconnects. That said, we've had a few of our internal users lose their connection to the web socket, and the app just beach balls. You can't do anything. Another example we keep coming back to is a user going onto a subway.
How do you envision spotty connections being handled long term?
If an app can be reduced to just CRUD, I think it works very well.
LiveView adds a `phx-disconnected` class, which is what I assumed you mean by beachball, and interactions are paused while the user awaits the UI to tell them they are good. This is all driven by your CSS and you can also hook into the connection life-cycle with a few lines of JS, so the app should be telling the user "Reconnecting..." vs it appearing broken, so this is really up to your UX folks. By default new phx apps make use of the topbar library so any page loading event or dropped connection will show the loading bar/spinner up top.
As far as spotty connections go, when I try simulating 30% packet loss, my LiveView WebSocket connections have no perceptible degradation, so it's hard to say where the cut off is. You can also fall back to long-polling which may benefit very edge users, but I would only do so if absolutely necessary.
Having said all that, one thing we are upfront about in general is LiveView is obviously not a good fit applications that require offline support :)
In principle, your Twitter client could remember the state locally, let you keep working, and just keep trying to sync.
In practice... most apps are really bad at working offline, so just handling the disconnection and reconnection in a robust and consistent way is already much better than average!
I think that 'read-only mode' and 'can't interact at all' mode are not the same.
I often have google news open when the train vanishes down a tunnel and keep scrolling down reading headlines until we're out the other side.
If a website freezes on me, I close it.
> Having said all that, one thing we are upfront about in general is LiveView is obviously not a good fit applications that require offline support :)
Not a good fit is rather charitable...
It becomes totally unresponsive and totally unusable in any offline or high latency situation (trains, stadiums, remote areas) right?
I know, and I've read your responses that 'well, all websites have to interact with a server eventually, so it's pretty much the same as that only better...' but, well... when you build websites like this, that's why product managers say "no, we don't want a website, we want an app".
s/obviously not a good fit/non-starter :)
> I think that 'read-only mode' and 'can't interact at all' mode are not the same.
I agree, but it depends on the application. LiveView doesn't "freeze", but the content on the page is not going to continue updating or be interactive. This indeed limits some applications that want to allow the user to continue editing a document, but your example of a news site absolutely still functions fine for read-only offline.
> It becomes totally unresponsive and totally unusable in any offline or high latency situation (trains, stadiums, remote areas) right?
Yes, just like the vast vast majority of web applications today, including the vast vast majority of SPAs that could, in theory, work offline, but don't because of the added complexity on the client and server, state syncing, conflict resolution, etc. If working under the offline condition is a hard requirement, LiveView is out full stop. But even for SPAs, this is an opt-in feature today that few choose.
2. What's your dev environment like? VSCode? What extensions, etc?
3. Excited to see you at fly.io! I think I saw a tweet about them wanting to make deploying Livebooks (built on LiveView) easy. Any news on that front?
Our goal isn't to replace SPAs, but I do think we'll obviate them for a large class of applications.
2. VSCode with VIM keybindings. I bounce from vim/emacs/vscode (all w/ vim emulation), but vscode has stuck pretty well for the last couple years. Much to my chagrin, it's faster than terminal emacs.
3. No news quite yet, but we're working on it. Stay tuned!
A someone who's familiar with Kotlin, I can see how Erlang's processes and mailboxes would work. The question I have is, you mention Phoenix maxed out available FDs but RoR would have struggled. I didn't quite get what those fundamental differences are that prevent RoR/Ruby from scaling-up io-bound workloads as effortlessly as Elixir/Erlang, given you point out that sync.rb was built upon a similarly capable event-io lib.
Also, if I may, what do you make of your former employers 37 Signal's turbo (hotwire) with RoR? Does the vision you have for Phoenix with LiveView match what you see them doing with hotwire? How do the solutions compare if you have had a chance to take a look?
Thanks.
Hotwire is building stuff from regular HTTP requests (ala AJAX) and uses WebSockets only for Turbo Streams. LiveView is all on WebSockets.
You can think of Hotwire as being stateless and stateful when required. LiveView is stateful.
Hotwire exchanges HTML snippets at various length whereas LiveView tries to do a very minimal diff for an update.
WebSockets would be faster for your regular updates, but you need to keep a connection open. In other stacks this might not be ideal, but it's what Beam is made for.
A simple case for me is using hotkeys to submit a form. Couldn't figure out a way to trigger a phx-submit from a onkeydown handler.
this.el.addEventListener('onkeydown', function (event) {
this.el.dispatchEvent(new Event('change', { 'bubbles': true }))
}, true); //<input type="text" name="body" phx-hook="SubmitOnEnter"/>
let Hooks = {}
Hooks.SubmitOnEnter = {
mounted(){
this.el.addEventListener("keydown", e => {
if(e.key === "Enter"){
this.el.form.dispatchEvent(new Event("submit", {bubbles: true}))
}
})
}
}
...
let liveSocket = new LiveSocket("/live", Socket, {hooks: Hooks, ...})It seems like your changes aren't really committed until they hit the database, but there are a lot of intermediate states.
We also have a hiring project that's designed to ease people in to Phoenix + LiveView. We extracted this from a real app and tried to make it as simple as possible to work on: https://github.com/fly-hiring/phoenix-full-stack-work-sample
Or can LiveViews be done on more performant languages like Go, Rust etc? I am just surprised why we don't see more LiveView implementations in other languages?
X probably doesn't have the mainstream appeal some people think it should have.
1. The concurrency and distribution model. Processes (light weight green threads) and extremely cheap, isolated, and concurrent. Process can message each other, and messaging is location transparent. So you can send a message to another process on another Elixir server using the exact same primitives as sending a message to a process on the local server. This allows for all kinds of things that are hard or not reasonably possible in other platforms:
- Start a process on a node in us-east1, and message it from Tokyo. This is simply built-in.
- Run a primary DB in us-east1, and RPC from your Tokyo instances to perform writes. The RPC mechanism is again built in. You have code running on another node, and you simply run your code over there and get the result. There's no marshaling of data structures, deploying message queues, protocol buffers, etc
- Using Phoenix Pubsub, broadcast a message from anywhere in the cluster, `PubSub.broadcat(:my_pub, {:new_msg, "hi!})` and it will arrive to all instances who are subscribed, anywhere
This kind of stuff can be made to work well in Go and Rust, but you need to bring in libraries and do more work. They are absolutely great at "network programs", but lacking the distribution primitives means libraries and solutions are usually more bespoke vs Elixir where everyone in the community simply uses what is provided out of the box. So there is no interplay of ops or dependencies to try to reconcile.
2. Processes support stateful applications. Most of the LiveView-like solutions still go over stateless HTTP because websockets and cheap concurrency aren't as viable. This drastically limits what you can do efficiency wise. For example, our diffing engine requires state to live on the server so we know what changed. If you simulate LiveView over HTTP by sending all the state from the client for every interaction, you are sending way more data, doing all the HTTP authentication, fetching the world, then sending the entire template back.
3. Process are preemptively scheduled and load-balanced on IO and CPU. This allows your LiveViews to perform blocking CPU bound work and the scheduler will make sure every other user gets their fair time share. In other languages like Node where you rely on evented IO, any CPU bound work blocks the entire program.
4. Processes are isolated. On the evented IO example, imagine your websocket handler in Node.js has an uncaught exception caused by a single user interaction. It brings down the connections for all connected users. In Elixir, all processes are isolated, garbage collected in isolation, so you aren't jumping thru hoops to handle these kinds of degradation modes.
Hope that helps!
At a previous company I worked for, a decision was made to move to other platforms not because Elixir was bad, or LiveView was bad (it was good!) but because it was really difficult to hire for.
It is difficult to find people with elixir experience, and we didn't want to have production system built by just people learning a language, platform and architecture on the fly.
Also, that's considering "backend" developers. It was *TOTALLY* impossible to hire frontend developers wanting to work in this setup. No single frontend developer wanted to learn Elixir, and all the backend related stuff. This doesn't happen for example with Node... yes, it might be a terrible choice for the backend, but most frontend devs are happy to work with it.
Regarding to the solutions in other platforms not having the Erlang VM to power them, etc,etc.... not everyone is at google/twitter/facebook scale, 90% of companies out there can be run perfectly fine with run of the mill django/rails/laravel setup.
Premature optimization is as bad when doing an unnecessary SPA as it is using the Erlang VM when rails was enough.
I would like the web app to have good offline support and use localstorage for backing when the connection goes down (re-upload when connection is back up). It it in anyway feasible to add some middleware to the client connection handling of liveview so that it still plays well with it's features?
I've read that it is possible to drop in some bits of JS here and there into the liveview powered page. Is it also possible to do it the other way around and go to drop in liveview "components" into an existing SPA? Throwing away my existing SPA code is quite a loss, but the liveview appeal is certainly there for other parts of the website.
Thanks for the great work on phoenix, it looks quite attractive.
I think it's really neat that Chris started out with real-time messaging. When I first started working with web frameworks, it definitely felt like real-time stuff was an afterthought—a layer built atop an older model that leaked a lot of the details.
Working with LV has been an absolute delight. Need to render a PDF and download? Fire off a `Task.async` to render in a separate thread on the backend. When it's done, that thread can just `send` a message to the LV process to update the UI that the PDF is ready. So easy and painless. LV really hits a sweet spot for me: I'm primarily a backend dev, but with LV I can build really nice stuff on the front-end with minimal effort/headache from NPM.
Basically I'm using it as a container wrapper for a large React app and keeping all the state on the client. This is only for the app pages on the site, the rest of the pages are either traditional "deadviews" or LiveViews.
Why?
1. I have a lot of state. Its an accounting app and I want the UI to be zippy. Because I don't want to keep all that state on the server (per user connection) I would have to use temporary assigns to keep the server state small which means lots of queries and data shipping when searching/reordering/generating reports. Nothing beats only shipping the data once over the wire. And I do know about all the hacks to update local lists using components - that doesn't help when I need to use/display the same data in different ways.
2. While I love the expressiveness of Elixir the lack of a type system makes UI development/refactoring much slower for me as opposed to React+TypeScript. Note: I have several years experience in both Elixir and React+TypeScript over many projects so I don't come to this conclusion lightly.
3. Using LiveView is much nicer than using Channels since I can delegate common elements of the page to it instead of replicating it in React. Sort of a Russian nesting doll of rendering. Plus the LiveView is colocated with the other pages in the site which makes it more tidy.
4. I don't have to write an API - its just LiveView messages.
5. I don't care about SEO for the app pages (and explicitly don't want it indexed).
6. I'm using Elixir/Phoenix/Ecto for its best parts - supporting lots of websocket connections and hosting the core logic (and the non-app pages). I shudder at the thought of running a fleet of node apps to do the same.
I'm not sure why I wrote this other than to let folks know that LiveView can be used in ways that might not be obvious from an initial look.
I seriously don't understand server-side rendering when toasters are more powerful than a mid tier pc i bought 15yrs ago. Why not do apis and just send the data?
I've been able to build rich, interactive, games without a single line of Javascript. It takes complicated server / api / front-end build projects and results in literally 1/10th the amount of code for the same result.
It's one of the few times in our world that the technology isn't just "new and cool" but "fundamentally better".
We were mostly split between Reactors and Gophers, and the amount of time we spent arguing about the API and implementing it... I'd say was easily one third of the word, if not more, that would have just gone away if we didn't have to worry about the details of the data we were moving back and forth.
My current job is a bit too front-end heavy for LiveView to make sense. But I still keep an eye on Phoenix. It really feels like it could be a secret weapon, at least until every one else adopts a similar pattern.
In React / Typescript world you can get a little bit of this with Blitz - but not the live updating part, as far as I know. I found there was quite a lot to learn, but I’d also been out of the React world for a couple of years. Probably getting up to speed with TS was half of it, and obviously that’s not due to Blitz.
It has felt quite magical thinking only about the DB and React, and so far it’s just worked.
“Blitz is a batteries-included framework that's inspired by Ruby on Rails, is built on Next.js, and features a "Zero-API" data layer abstraction that eliminates the need for REST/GraphQL.”
I do agree the batteries included part of Blitz makes it really pleasant to use if you need both front and backend.
I’m not sure I follow - are you talking about client-side invalidation, which you may have to do manually with functions like `invalidateQuery`? I can see that being a gotcha in more complex Blitz apps. https://blitzjs.com/docs/mutation-usage#cache-invalidation
This is one LiveView feature I've deliberately avoided so far. The reason is that when you replace real page loading with client-side navigation using pushState, accessibility for blind users suffers. When a real page load happens, a screen reader knows when the load is complete, and can handle it in an appropriate way, e.g. by beginning to read the new page. But when client-side JS updates the DOM in-place, a screen reader has no reliable way of knowing that conceptually, a new page just loaded. The usual work-around is to have an invisible ARIA live region that says something like "navigated to [page title]". That's better than nothing, but still a regression from real page loads. Of course, SPAs have the same problem.
This really ought to be fixed in ARIA, but until then, I'll keep doing form submission and page navigation the old way. Still, LiveView is really nice for real-time updates within a page.
def handle_event(save, params, socket) do
...
{:noreply,
socket}
|> put_flash(:info, "It worked!"
|> redirect(to: "..."))}
end
I'll also check out the ARIA region approaches. I think something like that would be drop-in for LiveView in the live layout, and we at the very least could include docs on it.
def handle_event(save, params, socket) do
...
{:noreply,
socket
|> put_flash(:info, "It worked!"
|> redirect(to: "..."))}
end
def handle_event(save, params, socket) do
socket = socket
|> put_flash(:info, "It worked!")
|> redirect(to: "...")
{:noreply, socket}
end
https://alistapart.com/article/the-future-of-web-software-is...
HN discussion: https://news.ycombinator.com/item?id=26265999
I find developments in this area very exciting as I'm sick of dealing with layers and the required tedious glue you have to write to join them together.
But you write only the code of the validation, and the vue, once, so it's more productive, and the state is always consistent.
I assume this means no PWA mode, although I don't really miss this.
Currently my ping to this website is 167 ms:
ping news.ycombinator.com PING news.ycombinator.com (209.216.230.240) 56(84) bytes of data. 64 bytes from news.ycombinator.com (209.216.230.240): icmp_seq=1 ttl=49 time=167 ms 64 bytes from news.ycombinator.com (209.216.230.240): icmp_seq=2 ttl=49 time=167 ms 64 bytes from news.ycombinator.com (209.216.230.240): icmp_seq=3 ttl=49 time=167 ms
Which means it's a cost I have to pay, before payload matters, before parsing matters, before rendering matters.
I'm at home, on fiber, on an ethernet cable, so ping should be very fast. And it can be:
ping 8.8.8.8 PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data. 64 bytes from 8.8.8.8: icmp_seq=1 ttl=114 time=5.63 ms
Unfortunately, the network is not uniform.
It's of course, worse on Wifi, and even worse on mobile phone, on the go. Depending on your location, YMMV quite a lot as well. Then you have hardware quality playing a role, the user may use other software (torrenting, streaming, playing) that can also affect that.
Bottom line, you cannot assume the internet connection is snappy. Or stable (which is a problem with websocket, as you have to implement robust automatic reconnect, and it is slow and expensive, but then your use cannot use your software).
So knowing the trade off you make is important. On a local network in the corporate world, life view seems like a terrific deal. On a heavy mobile user centric app, it will probably hinder your app ergonomics.
It doesn't mean that you don't use javascript at all, you just use it in a limited fashion for enhancing specific bits of your website. Just like it was in the 2000s when making websites was easy.
Then, if something modifies the data in the database not by using the Phoenix app, the Phoenix app would not find out until it loaded the values from the database. And what would prompt it to do that?
But if your entire active state can fit in RAM in a cluster of BEAM VMs, you might turn the BEAM VM cluster itself into a distributed database, in which the only way to modify the data is by talking to the Phoenix app (using Liveview, regular REST HTTP API or something else, maybe even postgresql protocol). If this is the case, the app server can guarantee that no state change could have happened by something else modifying the database or whatever, and a client receiving updates via websocket would be sure that it has a complete and accurate picture of the data.
Of course, you would need snapshot isolation (versioning of tuples in RAM) and you would need to store the transaction log durably on disk and you would need to garbage collect (postgresql vacuum) the no longer needed records. Snapshot export and "live update pushing" to a traditional SQL database would be desirable for BI and stuff. But basically, if a modern distributed database was also a good application server, it would be neat.
The Materialize folks made a blog post/demo of the materialize->app server->browser pipeline in Python+JS you could follow along with. They have a Postgres CDC implementation too, so hook it all up and it will go.
https://materialize.com/a-simple-and-efficient-real-time-app... (the images are 404 though)
It's hard to imagine an active state that couldn't fit into RAM in a cluster of BEAM VMs. I've run clusters with thousands of nodes, with some nodes having 768 GB of ram. With today's servers, 4 TB of ram per node is approachable. The new pg module avoids pg2's dependence on (cluster) global locks that limited effective cluster size.
Of course, you have to want to do it, and mnesia is mostly key-value, so I don't think you'd have a good time if you need other kinds of queries (but I could be wrong). And you need to have readily partitioned data if you want your cluster to be operable; having all the nodes in one mnesia schema makes a lot of things hard.
Just like Rails, phoenix is the doorway to your application. If there are data changes happening that aren't through your application, then you're doing it wrong.
The smallest piece of the pie possible. Even if you ignore things like wordpess.
This is super-unrealistic. State mutation legitimately happens through many channels. The trick is signaling to the application layer that state has changed and either the app needs to reload it or update it.
In the case of GP, I would either build in a web callback that can be used by outside processes, or put a message on a queue like SQS that can be consumed by the application. This isn't a phoenix thing so much as an enterprise app design thing.
13 years in professionally, many companies, many contracts, and many projects later I've yet to see a system where it wasn't the case. Small or large (16k/rps at the large end). Startup and "enterprise".
And why wouldn't it be the case? Why not have your data flow through a "central" business logic layer?
It sounds like Stimulus Reflex is essentially Sync v2 (today's libraries are able to accomplish Chris's original vision)
A Big Phoenix idea though seems to be that you can take this a big step further: now that you're syncing, keep the state that you'd be attaching to React components serverside, and let sync update the front-end. It feels like a lot of the benefit you'd get out of a carefully-designed GraphQL API, but with none of the plumbing work.
Many web front ends are built on the Elm architecture / Redux or my favourite, re-frame in Clojurescript land where the view is driven directly by the app state.
What I'd love to have would be the same LiveView connection, with one part of the state object on the server synced to one key in the global app state in the browser. You could keep the rest of the server data private, and the browser side can keep it's own state that's not needed on the server out of the way, but one part would always remain up to date.
Not sure if it would be 2 way sync, or more likely, send events back to the server and the updates would magically arrive on the syncd part of the app db. Events from server to client would also be useful.
It should be pretty efficient if it was diffed too. Phoenix LiveSync ;-)
One thing that concerns me about doing everything over WebSockets, is that it seems you now need to keep a connection with every connected client, even if they are not doing any "highly interactive" actions. I think (in most cases) it would be more efficient to do a bunch of AJAX requests to receive those HTML chunks. Now that we have CDNs, HTTP/{2,3}, etc. the benefits of using WebSockets for everything seem less obvious.
This is what HTMX[0] does by default, and then you can use WebSockets[1] if you need it. Another great thing about HTMX is that the backend can be whatever you like, as long as you can handle HTTP requests and return HTML.
In any case, I like both approaches and I would love to see the web development ecosystem moving back to sending HTML over the wire, regardless of the framework.
But seriously, what is the problem with this? I am guessing you don't really have much experience with the BEAM. It's completely not-objectionable, there aren't many stability issues, and they are very light weight (maybe a couple of kB/process).
I believe (have not done this myself) you can do liveview over long-poll, since liveview only cares about there being Phoenix Channel, which abstracts over websocket/longpoll.
https://equip9.org/2020/01/06/phoenix-liveview-longpoll
One of the things I want to try for the gee-whiz factor is to build an Elixir WebRTC client and serve liveview over a WebRTC data channel. I see no reason why that shouldn't work.
SPA/React/Flutter are great tools and technologies, when you have a team for the backend and one for the front end. If you are a little startup or a single man project, that’s quite huge to maintain
As you say SSE seem more suitable for notifications rather than bi-directional communication.
This is a pattern that I started using for my Liveview pages: https://www.youtube.com/watch?v=HA4h0cajgaA
Not much help if you are looking for full LiveView apps, though. I haven't checked it out yet, but i hope surface addresses some of the structural organization issues.
Phoenix, while it does have opinions, is far less opinionated in the sense that it doesn't do its darndest to force you into certain conventions (for example, if your module name doesn't match your file name, Phoenix won't complain). Its generators do try and push you toward using good DDD practices (which is my opinion is a GREAT thing), but of course the generators are completely optional.
I don't have experience writing large LiveView apps but I would say that if you are familiar with any component-based frameworks (like React), I would take a look at SurfaceUI[1]. It simplifies a few "gotchas" in LiveView (though I would say they are very minor gotchas and worth learning about at some point) and gives you a component-rendering syntax more like React. Once you get going, you'll learn that LiveView doesn't have all the headaches that come with bigger React apps (like having to memoize functions or comparing props to avoid a re-render and whatnot). The recent release candidate for Phoenix 1.6 has made strides for a cleaner component syntax, but if you're having trouble with LiveView, Surface might bring some familiarity.
https://docs.google.com/spreadsheets/d/10gCxVJyrme6Rv-LepqId...
Once there are enough contributions, perhaps a community member could write a blog article summarizing the major patterns - the use cases, patterns, benefits & drawbacks.
Don’t be shy!
(I’ve become a big fan of Elixir as a former Smalltalker)
With that being said, it's hard to replicate LiveView in most tech stacks in an as performant way. Blazor feels like LiveView at a glance but doesn't feel like it could handle as much as LiveView in production. The key difference is BEAM was designed like an operating system (processes, preemptive scheduling, etc) instead of a programming language runtime.
It being stateless is normally a feature, as it doesn't matter which server the request hits, we can elastically scale them up and down.
With a "session" being server-side statefull, you now have session affinity and are you persisting/hyrdating that session somewhere? Is it kept in memory and lost of the server goes down? How long does that session persist? Lots of questions...
Currently on .NET 5 the missing of hot-reload is the biggest issue. Otherwise its really a breeze to build anything.
It's been amazing to work with.
Thanks LiveView for inspiring Livewire!
It saves development time and solves lots of other problems along the way. I think it makes better web apps just by the way it works.
We are been using it on all our projects the past year.
Alpine js is a great compliment to Livewire.
They call it the TALL stack. Tailwind, Alpine, Laravel, Livewire.
The performance is as good or even better than vue apps.
https://htmx.org/ https://alpinejs.dev/ https://turbo.hotwired.dev/ https://stimulus.hotwired.dev/
Compared to Hotwire:
I think hotwire is the best option if you're using Rails. From what I know, hotwire needs a lot more of backend "collaboration" than Unpoly, so you will need backend "hotwire" implementations. You can see these things are being built for django, etc.
Compared to Htmlx+ Alpine:
Let me start with the disclaimer that I think Alpine.js is an aberration. Writing code inside html attributes is a fantastic way to break every editor templating language plugin, syntax highlighting, etc.
Despite of me not liking Alpine, I think unpoly handles a lot more use cases, and it feels a lot more "declarative" to me.
Compared to both:
Unpoly is kind of the "django" or "rails" in this category of tools. It gives you a lot of stuff. From page transitions loading progress bar (a'la turbolinks) to ajax replacing links, to modals, to popups to sidebars, to automatic blocks replacement (up-hungry) to form posts, to handle error responses, layers, polling.... and it has an awesome way of writing your own "components" with the up.compiler api.
I honestly think the only reason Unpoly isn't the most popular solution in this space is because of bad marketing (well, no marketing at all...). I really think it is the best (if you're not using Rails, then I'd use Hotwire+Stimulus+etc...).
Meteor is not server generated but rather a SPA framework with a server side part to it. It helps you build a SPA app quickly and it also uses websockets for all the transport but usually you just send json data there.
I don't really like Meteor because the performance is not that awesome and there is problems with node since it is single threaded. Meteor still doesn't support worker threads or other similar solutions in node so if you have something blocking on the server side the entire app will go down.
Liveview in the other hand is server side only (except for the liveview part). All the state is on the server and when the state changes it will push out the changes in the template and rerender automatically. In Meteor the recommended stuff to use is pub/sub and meteor methods but you still have to rerender yourself just like in a normal SPA.
Meteor is thus simply a way to quickly build SPAs and Phoenix and Liveview is a way to quickly build SPA-like feeling to apps but with everything living on the server.
You can think of Liveview as Google Stadia but for websites. Everything is rendered on the server and you just get the updates.
What's the resource cost ?
And that adding an element on a list don't show it up before a server round trip ?
What's the latency cost ?
I have to wonder if a Phoenix / LiveView activityPub server+client could be built that would be compatible with it. Something that would appeal to existing Elixir devs.
I had long term goals to improve it but, as long term goals, especially when you're alone, isn't improving. And even more given how much manpower was throwed at it by its sponsor and it (personal opinion) went too bad to improve its codebase.
Regarding ActivityPub Servers. It's actually quite easy to build one -- but building one that is compatible with the current state of the fediverse is another story. Mastodon built on OStatus, then moved to AP but while keeping weird extensions and so on, complicating (a lot) all of it. There's an attempt at this on Bonfire[1] which is loosely a Pleroma fork (started at Moodle's MoodleNet, forked by their workers).
You'd have better luck building a "Mastodon" client (which would work with Pleroma's client API) than wasting your time working on AP S2S for Mastodon.
> We're currently in the middle of a refactor to convert all components and templates from LiveView to Surface, which is a server-side rendering component library (built on top of Phoenix and LiveView) that inherits a lot of design patterns from popular JS framework like Vue.js and React, while being almost JavaScript-free compared to common SPAs.
How far is this away, roughly? It's literally the sole feature I need to make the switch to literally any new (to me) web framework right now.
1. How does this stack handle non-browser clients, like mobile apps or APIs?
2. Can this scale horizontally? i.e, can you throw in more servers when you need to? Since servers hold state in-memory (IIUC) I’m not sure about this one.
The advantage is stack simplicity. As stated in the beginning of the article, LiveView completely removes your need for any kind of API between your front and backend. It also makes it very easy (through JS hooks) to offload any interactions you want to the client if that makes more sense. But of course, if you end up offloading EVERY interaction to the client, you should be using a frontend framework--LiveView is very clear about not being suitable for every need--if you are building a super UI heavy app (like a text editor or painting app or the like), LiveView probably isn't going to cut it.
I will pay the cost of a round trip if (a) it simplifies my life and (b) the cost is low enough. LiveView simplifies interactive app development (for me). Since I can run my Phoenix servers close to people, the round trip cost is usually very low.
Now if you want to avoid round trips to the server for interactions such as opening a modal, you can!
There are at least two ways to handle this - Javascript hooks that let you attach JS to your DOM, or the light AlpineJS framework. The latter is a perfect fit for Liveview and it's part of the unofficial go-to stack name PETAL - Phoenix Elixir Tailwind AlpineJS Liveview.
Every interaction will take a minimum of 200ms to complete, which would become fairly noticable.
For things that should happen instantly on the client, you'd run some JavaScript on the client via a pcx-hook (JS escape hatch), or using a tool like Alpine.js to handle purely client-side interactions.
I’ll have to try it out sometime now ;) there is still some nagging sensation that I’ll run into a fairly major blocker somewhere, but right now I can’t articulate what I’m afraid of.
I ask because I feels like Phoenix is rallying around the stateful approach of LiveView and just want to ensure that model isn't the only model of development Phoenix will support (short and long term).
One of the biggest perks to the BEAM VM is how well it handles managing state though.
Most other languages aren't designed to do this well, so stateless web apps focus on just passing information from the client to somewhere else that holds the state (database, redis, etc).
Nothing is really stateless, it's just a question of where you decide to hold the state.
It is so cool that this just has a small js layer client side.
> Alternatively, Blazor can run your client logic on the server. Client UI events are sent back to the server using SignalR - a real-time messaging framework. Once execution completes, the required UI changes are sent to the client and merged into the DOM.
Unsure on the Rails / Ruby side, maybe stimulus-reflex, but there is also Hotwire as well ofc.
Would the experience be enlightening regardless if I choose to deploy apps with it and use it at work?
If you are reaching for Django Channels in your Django project then yes I would investigate Phoenix, Phoenix.Channels, Phoenix.Presence, and Phoenix.LiveView.
Are there any good libraries like devise and activeadmin now ?
There's one small typo:
> Uploads us the existing LiveView WebSocket connection
"use"
I remember and I disagree. It was dog slow, run by a guy who made slides that said "Fuck you." and was rife with memory leaks.
I admit I came into researching Rails a little reluctantly and pretty early on, as well. That probably biased my opinion of things a lot more than it should have but a few tutorials later, and after fooling around with a toy project over the weekend I thought: "Neat. But either it's daaamn slow, or I'm doing something horribly wrong." I don't remember what I loved/liked but I remember being annoyed at odd laginess[2]. The more I investigated within the community, the more I felt like I was corresponding to "a bunch of kids". And I don't mean "kids" as in insulting-term-used-for-early-20-something-know-it-alls[3], but ... like 12-year-old boys trying to be popular, and failing -- a bit immature with unnecessary drama. I ended up not exploring much beyond that weekend. It's not that I found the community to be entirely unwelcoming; really seemed like it was. It just wasn't a group I was positive I wanted to be a part of.
Of course, not too long after that, I recall discovering the term "Brogrammer" loosely tied to Ruby on Rails and I thought, "Yep! That's what I meant by 'kids'". I remember reading a rage-quit post from someone that hit the front page of HN[4] a little while later, and a few other things here and there. It seemed at a certain point, the things written about Rails that found my eyes were more frequently some form of drama/nonsense than anything telling me why I needed to look more closely at Ruby on Rails.
[0] Miss ya, Lou, if you're out there!
[1] And not in any way a middle-management guy -- he managed multiple development teams over the years and was put in charge of probably the hardest job in his career, managing 6 people with critical, mostly different, responsibilities.
[2] My apologies for the vagueness, it's out of concern that faulty recollection will result in me maligning something incorrectly. I did these exercises monthly, for about a decade. This one lasted about 5 days because I decided it wasn't worth further effort; I wasn't going to use it.
[3] Yeah, I was absolutely one of those and have found that calling out others faults is really stupid when you share them.
[4] Zed Shaw; not sure where the original is but if memory serves, calmer heads prevailed after a short time, he took down the original and I didn't Google so I'll shut up and share the link I found: http://harmful.cat-v.org/software/ruby/rails/is-a-ghetto
Yeah, you could say “no, such overt manipulation is definitely unethical”.
But on a more technical side, I like to be a progressive enhancement purist (though sometimes it’s not the pragmatic option) and I don’t see any good reason why client-side scripting should be required for an online store. Can LiveView be used as a progressive enhancement, producing traditional HTML with links, form submissions and all that will work without client side scripting? If not, I don’t think LiveWire is suitable for that specific sort of site; and if it is, then surely you can’t know how many visitors there are on a given item? To be sure, the estimate will still be more accurate than the likes of (30 + 20 * Math.random())|0 which is maliciously manipulative, but it’s not correct any more.
I think that's the issue with Phoenix. An actual opinatred vision like the Rails one came as an afterthought.