Farewell to Regular Web Development Approaches
devo.ps
devo.ps
This used to be called Service-Oriented Architecture.
Before that, client-server computing.
Before that, online terminals.
No, it's not exactly the same. History in our industry, as Mark Twain said, doesn't repeat. But it sure does rhyme.
My main concern is that people have a problem with Completely Legitimate Architecture X, and then stumble on Completely Legitimate Architecture Y. What was a nightmare in CLA-X is a cakewalk in CLA-Y.
Then come the blog posts (previously the CACM articles). The books. The conferences. The whole grand crusade springs to life and promises to evict the horrid evils of CLA-X, now that the scales have fallen from our eyes.
Then after a while, CLA-Y becomes the norm and CLA-X fades into infinite obscurity. CLA-Y is unquestioned, unexamined. Just how things are done. Why? Because.
Then one day a bright programmer notices that if they rearrange the pieces like so, they solve a number of well known problems with CLA-Y. Not improve, not ameliorate. Solve!
And somewhere, a 32-year-old curmudgeon called Jacques Chester begins to write a long post about how this is actually just CLA-X, with a different name ...
Rendering html on the backend was the only option for years, as the clients didn't have the computation power. Now, we're getting to the point, where rendering client-side becomes possible.
We're not rendering html client-side for fun. It makes the UI more responsive, while keeping the overall architecture more simple. There's a cleaner separation between the client and the server.
We can (and we do) have separate teams working on client/server. It wasn't that easy before.
I'm 33, if that makes my comment more legit ;)
I highly doubt so.
In early 2000s desktop browsers were already quite powerful, ran Flash applets and had XMLHTTPRequest, so client-side webapps were perfectly technologically possible. Yet almost everyone generated HTML on the server side.
I even remember one webforum using pure XML "pages" and XSLT on client-side to build HTML. It's just that this overall approach (client-side apps, not XSLT) wasn't popular for some reason, until recently.
1. Because about 90% of web users were on awful Microsoft browsers which was extremely slow at processing JavaScript, and made a very bad job of rendering CSS. Now, 80-90% of users are on a modern browser.
2. Google's methods of indexing and scoring content changed - it now parses JavaScript, gives value to speedy pages, is good at detecting keyword stuffing, etc.
3. Always-on internet is now pretty much a given, even on phones.
4. There are more developers, trying more things, sharing more experience.
5. Tools. In 2006, we had Drupal, Wordpress, Plone, and a few other odds and ends. These days, we have Varnish, nginx, Jekyll, CloudFront, MongoDB, etc. There are a lot more options, without having to spend a fortune or build everything yourself.
As it so happens I'm working on an application with this sort of an architecture -- lots happening on the client, with command objects passing back and forth. So I'm not saying it's pointless.
If I have a point, it's that this isn't as revolutionary as it might at first appear. And if there's a larger point, it's that as an industry we have the collective memory of a goldfish on crack.
Which is probably more a reflection of our age-ism and the fact that 'hot new approach' is typically embraced as 'hot' and 'new' by people who were in primary school the last time it was popular.
That said, as long as people are throwing out 'the way' for something else, I think that counts as a revolution. Regardless of whether the new thing is actually new.
Neither democracies nor republics were particularly 'new' in the late 18th century. But their popular re-application was still a pretty big deal.
Except the time to first user activity is delayed. Speed is an important criteria to responsiveness.
The architecture problems can be solved without pushing it all to the client. It's a standard service-oriented architecture. Where the processing happens is mostly irrelevant.
Not if the client receives the requested page pre-rendered. The client should then render subsequent pages.
Everyone I know who is doing it is doing it because "that's the future". No logical or technical reason at all, just following the current fad.
>It makes the UI more responsive, while keeping the overall architecture more simple
It does neither of those things. Most javascript apps are much slower than the equivalent app using server side rendering of html. Look at how incredibly slow and clunky twitter is for example. The architecture of a typical fad javascript app is a nightmare compared to virtually anything else, super crazy "enterprise" java monstrosities are the only things I've seen that are more complex.
>We can (and we do) have separate teams working on client/server. It wasn't that easy before.
It could have been, we have that. Our client team does html, css and javascript. The server team does haskell and postgresql. The only thing you need to do this is a template engine that separates code from markup.
But technology did change. Faster upload and download speeds, camera phones with video, consistent browser APIs that rendered video more quickly. And now YouTube worked, not because it was a new idea, but simply because the technology necessary for it to work now existed.
Same could be said of static HTML. With a combination of Angular, Jekyll, S3 and websockets you can have a very quick site. This is not just a matter of fashion, but also the supporting technology.
All static. I could have deployed the app on a thumb-drive. And it had advanced searching, a book listing and recommendation system, a fully-configurable countdown clock for your startup, and a few other things. It got to the point that whatever I could imagine, I could imagine building it as a static site. And I would prefer doing so.
This is truly separating the view from the data. The same back-end services (for hn-books I didn't even have a back-end!) could plug into zillions of variously-looking front-ends. And as it turned out, the Query String was a heckuva lot more useful than I imagined when I started.
Neat stuff.
ADD: One more thing. If, like most folks, you have a class of data that doesn't change very often, say lookup tables or in hn-books' case the actual database, there's no reason you can't deploy that information as a static file as well. You can pull JSON just as easily from a static file somewhere as as server, and it uses a lot less resources. For my newspaper23 site, I used a combination: 4 times an hour I rebuild the database and deploy as a JSON. Then the clients all just pull from that file. When combined with a CDN this has tremendous scaling potential. And it's very simple to do. Why thrash the database every time somebody visits your site when it's just returning the same stuff?
Jekyll can be pretty damn slow though (especially on Github pages) so be careful with the size of your data set.
I think though that the control over the code base wasn't much of the issue: the mental bloat it was bringing in our thought process though definitely was. We many times solved problems following the underlying CMS' patterns, and that ended up eating more and more of our resources.
IMHO, our main gain here is the fact that we can focus back on the end user and the story we're trying to tell through our apps.
"But wait until you see the light! Software today is intelligent! Asynchronous front ends generated from esoteric scripting languages running on clusters of distributed virtual machines around the globe! Death to repetition! It’s more productive! It’s scalable! It’s fault tolerant! I followed the light and it only ended in darkness."
I also don't understand comments like "we can make drastic changes in the UI without any change on the backend"
In Wordpress or a Rails App, I can change the entire UI without having to touch any "backend" code (controllers, models, functions, etc). Am I missing something here?
All they've done is to move a chunk of the backend into the browser. It's still there, running on an untrusted environment.
This has always been possible with well-architectected systems, but more importantly it's mostly a platitude. It just doesn't happen in real life. "Drastic changes" to the UI are almost always driven by changes in requirements and hence they impact the entire system.
With a clean separation of client and server, we've been able to deal with these changes more easily, being able to prototype something HTML5 + CSS and quickly transforming this prototype into the real final product. Working with fewer layers in between you and the actual code (effectively HTML5 + CSS + JS helped too.
I'm not questioning your ability to be proficient at it with a CMS (or Rails). I am much more comfortable with our current approach though.
Of course there may be more pressing issues mentioned in the article, but I couldn't read it as 20% of every single line is cut off.
I really think it's important tha
and approach. When it isn't consi
class. After all, there are only
size.
1. What is a "static client" in this context? From my understanding, a static client is a non-JS web browser that does one thing: render HTML code that is composed and served by the server.
2. People here talk about "(not) rendering HTML on the server". What's that supposed to be? I thought "rendering HTML" means that you take HTML and display it on a screen, which happens ALWAYS in the client and never on the server.
Anyone care to explain?
"Static client" here doesn't refer to the user's browser. It refers to a front-end that serves web requests. It serves them statically, in that every URL maps to a static HTML file (with embedded javascript). Once the user receives that static HTML browser with embedded javascript, this then becomes the client. The javascript (run from within the user's browser now) communicates with a back-end or server, which the article describes as a "RESTful JSON API". It fetches JSON documents and uses the results to render HTML. "Render" here means "turn the backend data into HTML output".
This all is in contrast with a more common approach to dynamic web-pages, in that every URL request prompts a dynamic process on the server to do some logic (fetch data from a database often, and parse and interpret request parameters) to render HTML which is then sent back to the user.
Hope that helps.
2. it is about where the html is generated, either by js on client or by a script on the server.
Answering your second question sheds light on the first.
A better term would be "generating" HTML on the server — the browser is obviously in charge of rendering. In Rails for example, you can have ruby code in an ERB template which the framework compiles to HTML. Hence, the HTML is "dynamically" generated (on every request).
The opposite of dynamic is "static" where the HTML is either written by hand or generated just once (like Jekyll) rather than on every request. In either case, as a sibling comment mentions, an actual HTML file is served. However, thanks to JavaScript development maturing and JS engines getting faster, a lot of functionality that used to be done mainly server-side can be relegated to the client.
I personally wouldn't use the phrase "static client" much, though.
(Edit: Problem with "static client". To me a 'static' site means that it has no communication with a database, such as marketing sites or statically-generated blogs. 'Client' is too ambiguous as it's not necessarily a browser. curl is an HTTP client, for instance, and a JavaScript-driven "single-page application" would be pretty limited via curl.)
Someone also suggested C-z for undo, but no luck either.
I think I make that mistake fairly often. We don't actually mean rendering when we say that, sorry for the confusion. We actually mean the generation of html from whatever data we're displaying. So a server side template engine sends already generated html to the client, a client side template engine just gets the data from the server, and then generates the html using javascript.
There are tons "traditional" frameworks that are mature, proven to be reliable and are full of features.
I've tried about a dozen of both various so-called "RESTful API" (server-side) and webapp frameworks (client-side), and my overall impression is things are far from being mature. Primitive todo-list apps are easy, but more complex things quickly run into lack of features, documentation or just bugs.
It lacks a proper type system. It lacks proper support for namespaces and modularity. It is full of far too many inexcusable flaws ("quirks", as its advocates like to call them), even in such basic functionality as its equality comparisons. It's full of features that are outright harmful. Prototype-based OO has proven, in practice, to be inferior to the more traditional class-based OO approach. It's just one problem after another when using JavaScript.
CoffeeScript and other languages that "compile" down to JavaScript may help somewhat, but their JavaScript legacy is still generally quite apparent. Instead of being "good", they're just "not as outright horrible" as JavaScript.
Things would be much better if there were a wider variety of languages supported by browsers, including ones with the feature set necessary for effective, large-scale software development.
IMHO, when compared to asset compilers in many modern frameworks, the profits are disputable.
I've evolved to this approach - with the addition of WebSockets and a data binding layer (knockout) - to building the infrastructure for an algorithmic hedge fund over the last couple of years and it works extremely well.
Your backend components are free to do whatever they need to (in whatever language/technology) and publish data periodically. The web server simply dips into these streams of updates and transforms and routes them to the front end as needed. It's an extremely flexible and agile way to develop applications, I can't recommend it enough! In fact it's so nice to develop this way that I'm writing a book about the ideas and techniques I've used which I hope to post to HN in a couple of weeks when I've got the infrastructure and homepage up and running.
[1] Post: http://blog.ziade.org/2013/01/25/a-new-development-era-essay... Discussion: http://news.ycombinator.com/item?id=5120020
It starts with saying "well, let's just render html on the client-side instead of on the backend". Suddenly you're in a new world of programming.
http://www.tomshardware.com/news/google-googlebot-ajax-javas...
it renders each fragment to a file and serves that to the bots, right ?
careful to only serve these pages to bots tho
It's not as much pain as you would think, it's enough to just return some html. No css/js is needed.
Still, that's some additional work.
Probably worth reading this: http://productforums.google.com/forum/#!category-topic/webma...
They'll accept geo-targeted or mobile vs desktop optimised content, but everything else is a grey area.
Once of the reasons Gawker moved away from hashbangs is that they weren't well supported by Google, despite this being a "Google specification". Nick Denton, the founder of Gawker, has some interesting tweets about the use of hashbangs over the past few months:
Nick Denton: "Dip in uniques largely because of drop in Google refers. Pageviews (which are driven more by core audience) less affected." -- http://twitter.com/nicknotned/status/61152134929981440
Nick Denton: "Google does not fully support "hashbang" URLs. So we're eliminating them rather than waiting for Mountain View." -- http://twitter.com/nicknotned/status/61465859079671808
Nick Denton: "Yeah, I'd advise against hashbang urls. Will kill search traffic -- even if you abide by Google protocol." -- http://twitter.com/nicknotned/status/62595141927583745
So here's a high-profile failure of hashbangs when it comes to being indexed by Google. Which means if it's search indexability you want, hashbangs isn't it. The Google spec just isn't mature enough, or well supported by Google search itself.
I think it is more about do you want to figure out the html in JS or something else, and do you want to pay the bill to make the pages at home or lose some customers who don't have the power to do it themselves.
You can do what Twitter did. Allow some duplication and render html server-side on the first request.
They ain't no silver bullet: you have to decide where you want to invest your time.
It didn't last because, unsurprisingly, plain HTML loads renders a lot faster.
That said: XHR is a genuine change in the model. So too the leaps and bounds in bandwidth and computing power.
And JavaScript engines performances too.
What is your point? You write in italics that "plain HTML renders a lot faster" and then you contradict yourself.
Part of the confusion is that HTML is living a strange life as part document, part app platform. I mean if you compare HTML and JS to the fit-and-finish of a mid 90s platform it's a mess.
But, it's what we have.
I also think that there will be a swing back to an almost complete server-centric model after that, where all logic and rendering will take place on the server and be streamed to extremely dumb clients...like TVs with basic touch input. This will require a pretty intense infrastructure, but with some nice perks for hardware/software developers. For one, absolute control over an application...bump versions any time you like, easy access to debugging information (crash reports, etc.), watch user usage in real time, on and on. Hardware will also be insanely cheap. Users benefit from the practical equivalent of having a top-of-the-line computer embedded in their smartphones.
Most humanitarian groups want an easy to use CMS with some features, like donation modules, added in. Is the OP building out the content editors too? How is that not a CMS?
I've yet to work with a humanitarian group for which Wordpress was not more than they could handle on their own. When something breaks in their new-wave architecture, how day is it for them to fix on their own?
Edit: I realize the OP is talking about web apps, not just websites...WP is likely not a great option then. But that's why I'm asking for some examples of what they've done and why it justifies a "farewell" to old web dev
I still think there are relevant use cases for using a CMS. However, from experience I think that:
1. In many cases, CMS like Drupal or Wordpress are seen as the silver bullet for everything and anything; "Why use a 3rd party? Just built it in the CMS!". That's how you end up with expensive, tightly coupled platforms that are effectively locking these organisations in.
2. Many organizations are just engaging in some kind of arms race, copying what their neighbors and competitors have.
I completely agree with you that CMS is seen as an intractable silver bullet. However, I think it is too extreme to say that web development, as we know it, is out the window. While Wordpress is not the silver bullet, the ability to graft on 3rd-party APIs and client-side apps (even something like Disqus/Facebook comments) makes knowing just Wordpress still incredibly useful, for clients who do not need bespoke data solutions.
But even with a client who needs data-resources, it seems like Wordpress could still play a key part by being the HTML page server with javascript includes to an external data app...deciding to build a new framework just to accomodate unique data-resources can still be overkill.
"It’s not that we fundamentally disliked the previous approach. We just reached a point where we felt our goals weren’t best served by this model. With lots of dynamic data, complex visualizations and a set of “static” interfaces, the traditional way was hindering our development speed and our ability to experiment."
It's very much targeted to our specific needs. Our approach is no more universal than Wordpress or Drupal. We have however harvested interesting conclusions from moving away from these CMS and that's mostly what I was interested in sharing.
With regards to using Worpress as a middle man for data centric apps, this is the kind of things we've done. My opinion: CMS are good at doing regular type of content CRUD based apps, but it breaks quickly. Moreover, most popular CMS are becoming bloated pretty quickly.
Have a look at Jekyll (http://github.com/mojombo/jekyll), it's a much better fit for that.
Lots of people know how to work on Drupal and Wordpress, and it's not hard to find vendors willing to step in and take on the operation and maintenance of an existing Drupal or Wordpress website. In fact there are entire organizations--Automattic and Acquia--dedicated to providing such support services. This provides an essential hedge against poor vendor performance, which, sad to say, is all too common in our industry.
The goal to to completely replace the application with an API and several UIs, including a full web UI, a mobile UI, and also 'embeddable' UIs.
The API also serves as an integration point for other products in our suite. That allows us to scrape a dozen different SOAP services that are targeted to specific integrations and have those other products all utilize a single API that can do anything that the full product can do.
I'm really happy with this architecture.
...wait a minute.
CORS and session handling really are the main issue we faced. Easy to handle though once you understand the limitations.
It's actually quite simple to provide "static" outs for WordPress and the benefit is the client gets a nice easy to use admin. It's not the best thing in the world or anything, but it works.
Words have broadly agreed meaning, despite what you or Humpty Dumpty may think.
There was an article yesterday about how Airbnb was going through some serious gymnastics to get a fully rendered HTML page on initial request and then keep it interactive and client side rendered after that. Why bother? Just send back HTML all the time.
I worked on a very large (100+ developer) application that was fully bought in on Backbone + JSON api consumption. In the few cases where there was a lot of client interactivity (editable grids and drag and drop, most notably) the client side rendering approach made sense. For 90 percent of the application though, partial rendering of HTML from the server would have made things easier and less complicated (and faster!) The initial time to usable page is MUCH longer when you render static content, then make an ajax call to get the data.