Server Side Rendering at scale
engineeringblog.yelp.com
engineeringblog.yelp.com
The fact that they don't make a reference like, "hey, ya know, how _everything_ worked just a few years ago" tells me they think this is somehow a novel idea they're just discovering.
They then go on to describe a convoluted rendering system with worker processes and IPC... I just don't know what to say. They could have built this in Java, .Net, Go, really any shared memory concurrency runtime and threading, and would not run into any of these issues.
How things worked a few years ago: you wrote SSR pages with one set of tools (like Django Template Language), then hooked into it with another set of tools. If your pages are complex enough, you end up with weird brittleness because the "initial page load" is not handled the same way as modifications of that page.
Now it's much closer to using the same set for the initial load and subsequent edits. This is a net win for people working on the frontend, in theory.
The more nuanced thing is that frontend tooling is so lacking in terms of performance, despite being something that theoretically should work very fast. In particular, having a bunch of language tooling written in Javascript is the JS ecosystem's billion dollar mistake IMO.
Server-side web frameworks even have modern component-based UI templating now, and features like maps can be layered on top as progressive enhancement without this bloated frontend mess.
Also, the interface of HN works because of its demographic.
Much of the "progress" in web development in the past decade feels like it's just fixing problems that only exist because we're building web applications this way. React, Redux, Typescript, Server Side Rendering.. these are solutions to problems we created for ourselves by using an architecture with a dubious value proposition in most of our use-cases.
I have to wonder how much of the client-side rendering framework use is mainly from people forming a cargo-cult around "it's what Facebook does" when at the end of the day what Facebook does is way different from your use case.
By strictly numbers, most engineers don't work at one of those tech giants, but at small firms making CRUD apps that don't really need all that responsiveness.
And, importantly, a lot of those websites have complex admin-side backends to deal with a lot of things as well! Of course in those cases you might say that the UX requirements are lower (true), but when you are basically serving as a DB frontend, you want to be able to do things quickly, even on a really bad connection.
There's a lot of cool work into partials for SSR, but React and friends are also very nice because you can build out complex UIs that operate through an API, without having to figure out a bunch of workarounds for stateless HTTP/HTML (form wizards anyone?)
But Yelp is nothing like that. What part of the interface is so advanced? It's just a few links and buttons to navigate to pages and submit reviews. A few JS event handlers can handle AJAX/partial updates without an entire React frontend.
There's even stuff like https://alpinejs.dev/ and https://htmx.org/ to make this incredibly easy now.
"Advanced forms" come up all the time, and are usually when you have even a bit of non-trivial business logic. For example "if people pick this set of options, show this other option" (but you want to avoid having a form wizard, cuz then you're having to track state across multiple page loads). There's also stuff like in business applications, previewing calculations and the like.
"A few JS event handlers" can handle a lot of tiny things, but many B2B SaaS have a handful of pages with a loooot of these things, and at the end of the day you could have your hacks, or you could try to be principled about loading it.
Though it's also about just finding the right mix, there are people who can architecture their stuff "correctly", completely SSR + some JS flavoring. But it requires having people who are really good at backend and frontend figuring out the right pieces and putting it together. That diligence is hard!
The new cutting edge with Blazor is even more impressive and a serious contender for non-JS frontends. Similar advancements with Elixir/Liveview and Ruby/Hotwire
It is funny how you can get away with using the wrong tool for the job in Software in a professional environment. Like imagine if you were hired to build a house, and you decided to build the foundation out of modeling clay because that's what you're used to working with. And then you started to come up with novel methods of hardening modeling clay when it proved not to be fit for purpose.
I guess you can get away with it in software because these kinds of decisions normally only manifest as increased server costs, or moments of users' lives lost to performance issues, which are much less evident to the outside observer.
- sourced from local material (use a different type of clay or straw if you need), no need for chemicals or for sand imports from depleted beaches on the other side of the world - recyclable to almost infinity (need a bigger/better house? just tear it down and reuse the materials) - cool in summer, warm in winter if you design it well - lasts for decades if your structure is designed well: i'm not aware of really old examples but it would be a surprise if the structure outlives us all (does someone have resources on this topic?)
So turns out your example was more interesting and less absurd than you originally thought. Just like server-side rendering uses an order of magnitude less resources (for n clients it's O(1) with caching, whereas client-side rendering is O(n)), it turns out clay is the perfect material to use an order of magnitude (or even more orders?) less resources to build your house than if you used concrete source from various polluting industries and endangered sand deposits.
I'm a big fan of server-side rendering and am sure it could be used way more often to great effect.
> After a string of production incidents in early 2021, we realized our existing SSR system was failing to scale as we migrated more pages from Python-based templates to React.
Apparently their old server-rendered Python app (not called SSR though for obvious reasons) was scaling just fine before the migration.
> We evaluated several languages when it came time to implement the SSR Service (SSRS), including Python and Rust. It would have been ideal from an internal ecosystem perspective to use Python, however, we found that the state of V8 bindings for Python were not production ready
Wow, that is some convoluted architecture
It seems like the problem here is React
React is a defacto standard that product people know and understand. They can ship complicated features faster across platforms (for example React Native), while the more technical engineers get to solve these more generic problems (dependency hells, integrations, CI/CD, performance).
Alternative solutions would potentially require more technical engineers to build features and thus slow down product development for a big company.
Pointing out badly constructed solutions using the wrong tech is valid criticism. It's not "hate" to say that React and this SSR setup is used being used unnecessarily for a site that doesn't need it.
> "React allows you to release faster as a big company."
This is absolutely not a rule in any way. Again, using the proper tech for the situation is what matters. Large teams with complex frontends can sometimes move faster with React, but there are million other factors that go into this.
And for some strange reason this mostly happens to Web Development in general.
2. Watch potential competitors chase their tails & burn budgets adopting hot new thing
3. Quietly drop or sideline solution a few years later
Mind you, sometimes it's inadvertent. I'm currently the only web developer at my company building a big system in Go and React. I don't believe they are very difficult or esoteric technologies, but I'm still not sure if they will be able to find a replacement if I decided to move on.
But I don't know what the alternative would have been. Probably keep trudging on with the old PHP + Dojo bombsite, but it would have the same problem because who would want to work with that tech stack? Who would be able to be productive in a 200K LOC pile of shit? I mean even if it wasn't shit, it's still 200K LOC representing a decade of work, dozens of domains and hundreds / thousands of individual features.
Which is where technology choices come in again; use simple and few tools, the problem to be solved is difficult enough already.
I think it is because a lot of tutorials nowadays talk about how to do X with tool Y, without telling the historical context as to why Y is in use in the first place. Usually the tool is to solve a specific problem in mind. When people discover that the tool doesn't solve their very own problem, workarounds based on the very same tool is devised.
Other common examples are kubernetes and microservices. I have seen startup jumped onto the bandwagon before having real customers where the tool in question is meant for scalability purpose
Really? A technique invented for JavaScript?!?
Instead, this blog post shows they made small changes that resulted in a much better performing site with less server resources needed.
The idea of dropping server side rendering if the site is temporary overloaded is a good one that you can’t do if built in Java, .Net and Go.
A fork exec web server is not convoluted.
Getting the desired plaintext documents across the network has never been such a clusterfuck in my experience.
SSR improves the user experience for first time use of those apps and enables SEO.
What you say is subjective but I somewhat agree with your opinion, but only for web sites, not web apps.
Overall I agree Yelp should have stuck with the existing system as their product functions as a site.
Any .html page works offline. You don't need any JS or framework for that. If your JS-powered page doesn't work offline, it means either it requires online connectivity to solve problems, or its badly designed and does not respect "progressive enhancement" principles.
> SSR improves the user experience for first time use of those app
SSR improves UX for everyone. Seriously most "web" pages these days takes dozens of seconds to load and use a non-negligible percentage of our CPU/RAM. If you want to know what real-world conditions look like for literally over a billion people, run tests from a core2duo (or a similar VM) with 2GB RAM with simulated 10% packet loss and 1Mbit/s bandwidth.
Yes, people usually reach for an SPA when they just need a site, but SPAs have their uses.
Couldn't you built a native application and NOT have this problem? It seems like yet another self inflicted problem.
What's with other's telling people what technologies is best for them when they aren't in the scenario they are in?
Right, they'd have brand new issues because they were dealing with shared memory.
JS doesn't have an advantage here, it's just limited to being single-threaded.
Node.js is 13 years old now. I get that a lot of people think it's some new thing that only dumb frontend programmers who only know JS use, back in my day, grumble grumble. But at this point it's established, mature technology, es6 is an expressive language (much, much more so than Go, C# or Java). The multi process concurrency model is sometimes a pain in the ass and sometimes just want you want.
I've used ASP.NET and node.js in anger, have you? Or do you just attack it because you hate the JS of 2010?
> But at this point it's established, mature technology
I have to disagree. Day to day I maintain a variety of node.js, scala and kotlin services. By far, the most gotchas have been from the node.js services.
Simple things like validating incoming payloads are made ridiculously complex. You can get a library to do that, but which do you use? And then most of them don't export the typescript type signature so you don't get any type safety unless you re-define the type in typescript.
Let's say you want a tiered caching library? I've spent weeks of my life debugging concurrency issues because the memory store shared objects whereas the redis store didn't (because they were serialised and sent to redis). That's on the most voted caching library for node.
The quality of node.js libraries in general is far below that of scala/kotlin, and the maintenance cost is way higher. You can't easily tell if signatures for the apis of the libraries you import have changed as well. The ecosystem is still moving very fast, and while I agree that es6+ makes it a much better language, it causes its own issues when interacting with older code/libraries.
Expressive as the language may be, in production systems it leads to poorly understood code because of all the ways to do simple things, and the gotchas associated (something i believe it shares with scala).
> The multi process concurrency model is sometimes a pain in the ass and sometimes just want you want.
Give elixir a serious go and see if you ever say this again. Concurrency should be managed, much like garbage collection has been for 30 years.
I strongly recommend Zod[0]. You write your schemas in code, it validates incoming JSON at runtime, and you end up with either an error or well typed code. The schemas themselves are powerful (you can check properties of strings, etc), and the derived typescript type comes out for free and looks like a human wrote it. A very powerful tool, with a very intuitive interface.
Give elixir a serious go and see if you ever say this again.
Ah you hit me where it hurts - really need to seriously try the Erlang ecosystem.
Where did I say that?
I use C#, Go, Java, and lots of JS/Typescript. I like them all. I find C# far more expressive then ES6, but that's just subjective preference. The point is that the vast majority of web frameworks simplify everything to a URL route that runs some backend logic and returns some response (HTML/JSON/etc). Requests are already well isolated and you don't need to worry about threads and memory.
I can't even think of what shared memory issues a typical website like Yelp would have. Can you provide an example?
However if you do need to worry about complex multithreading, memory access and concurrency, then Node is a poor choice. The other language stacks are not only faster but have the proper data structures and ergonomics to handle it while Node/JS is single-threaded, requiring more work and creating more bugs.
SSR ... you mean like PHP did it ... and most all of the tech did it before.
How did this even make it to Hnews...
No real-time updates, no complex rendering, no serious in-depth application abilities. The only big JS feature is the maps which can be wired up with relatively little code.
Thinking that they haven't thought about this one simple way of doing it is a bit naive.
Yes, some things are just not that complicated. Sometimes it really is just poor tech.
Although to be fair <50 req/s/instance is pretty bad performance so a rewrite is probably warranted
With their new model, they distribute requests explicitly to workers so they can’t interfere with each other and drop server side rendering if they don’t have capacity.
[O] https://stackoverflow.com/questions/274290/any-big-sites-usi...
It seems way easier and faster to instead start as many node.js instances as you need and distribute the load via nginx reverse proxy.
It's unfortunate that ssr for client frameworks has to run in node.js, which is rather slow. But that shouldn't invite someone to stack more server javascript on top of it. Maybe someday someone is brave enough to write a frontend framework that uses a compiled language (rust, go, java?), that can also compile to javascript for the client, but renders crazy fast in ssr.
Depending on use case/workload, the bottleneck is mostly startup time. Message passing is much faster than most people think, and the main thread ideally is primarily just doing that and getting out of the way. Spinning up new processes is more expensive and so is IPC. Of course if you don’t need coordination…
> It seems way easier and faster to instead start as many node.js instances as you need and distribute the load via nginx reverse proxy.
Sure. If you’re running a stateless server. To be honest it’s not clear to me whether that’s the case here. But for other use cases where coordination is needed and which would benefit from concurrency, worker threads are often the best solution for Node.
> It's unfortunate that ssr for client frameworks has to run in node.js, which is rather slow. But that shouldn't invite someone to stack more server javascript on top of it. Maybe someday someone is brave enough to write a frontend framework that uses a compiled language (rust, go, java?), that can also compile to javascript for the client, but renders crazy fast in ssr.
There are lots of solutions in this space. Whether they’re worth adopting largely depends on their maturity and your team’s ability/willingness to adopt them.
But is it really concurrent though? Each request in the end is still processed by a single node worker which is limited to a single cpu core. Except you manage to distribute parts of the renderer to different workers in parallel. I tried that once as well, render the header, footer and body in 3 different workers. But in the end this consumed 2x as much cpu thus needing 8 workers instead of just plain 4 node servers. And the threads stepped on each other feet due workers being itself single threaded, so the 99%tile got worse/unpredictable.
All of my knowledge is of course biased by using angular, and trying to parallelize angular comes with some overhead due to dependency injection and having to spin up most of the app even though you may only want the header. So I could see how react.js ssr could indeed be split up easier. Didn't had the pleasure to benchmark this yet though.
Yes.
> Each request in the end is still processed by a single node worker which is limited to a single cpu core.
Even without worker threads it’s still concurrent, every async operation suspends and yields to the event loop. The difference is cooperative concurrency (the default Node concurrency model) and whatever concurrency you want with threads.
> Except you manage to distribute parts of the renderer to different workers in parallel. I tried that once as well, render the header, footer and body in 3 different workers. But in the end this consumed 2x as much cpu thus needing 8 workers instead of just plain 4 node servers. And the threads stepped on each other feet due workers being itself single threaded, so the 99%tile got worse/unpredictable.
Your workload very likely wasn’t CPU bound and probably benefited more from the traditional event loop. Most React use cases will be the same.
But pretty sure some engineers are having tons of fun building this stuff there. good for them and their CVs.