Hacker News Reader

There is one argument against dynamic sites where a static one will do: a static website is reliant only for security on the http server and the underlying OS.

Dynamic sites run so much code during page generation that there are a lot of ways in which you could try to use all that code to do something unintended. And some of those ways pay off.

A static site cleanly separates the moment of code execution and the moment of serving up the content in time, does not even have to happen on the same server.

So from an operational security perspective, if you can run a site statically you probably should. And what with blogs outsourcing comments to disqus and the like quite a few of them are candidate for being served up statically.

As for the performance element, if a site has 1000 pages then re-generating that site will be the same load as serving up 1000 pages when the caches are cold. So that's 1000 visitors. If that's slow then it is annoying to the site admins, not to the visitors. It's an off-line process, it doesn't have to be fast to work. All that matters is that the end product can then be served up blazingly fast.

Any website that does not require user generated input or logged in users is a really good candidate for this workflow.

> All that matters is that the end product can then be served up blazingly fast.

TFA argues exactly against this point. All that matter (or rather, what matters more) is minimizing the friction for the blog author in order to stay motivated, not if it takes a fraction of a second more for a page to load.

That's strongly dependent on how many people visit a particular blog. For smaller blogs, sure the page generation time is not relevant.

Indeed. For many people this issue is premature optimization.

When I tried to decide between a Middleman app with minimal configuration work and easy deploy (Heroku), or Ghost, Wordpress, or something like that, one consideration was page generation time.

I calculated that most optimistically, I'd probably write at most three articles a week (which is very optimistic). That means approximately 150 articles a year. That's pretty quick to generate, so I could even leave the generation process to heroku on first page refresh if I wanted.

I once made a static website with 12,000 articles. The key? * used a RAMDISK to generate the site and rsync to upload only the changed part to the server. * we also used JKL (a clone of jekyll in GoLang) * at some point we were doing 40 second compiles.

dynamic with caching (e.g. Varnish) is like static from security and response time perspective..but static has the advantage of local distributed fronts (cloudfront)... If we think, 99% of all blogs will be read less than 1000 times per day, the performance consideration does not matter.

I don't agree at all that 'dynamic with caching' is like static from a security perspective.

It's very easy to bust the cache (simply affix a random bit of nonsense to the end of the urls) and force dynamic execution of every request you make, opening you up to all the security issues that a direct dynamic front-end would give.

That's just a question of cache config surely.

Only if you have prior knowledge of what URLs are possible.

If you don't then the front-end cache will have to play it safe and will have to pass the request to the backend if it does not know the URL.

Any kind of variability in the request that causes a conversation between the front-end and the back-end and the cache can be busted, or is a potential DOS vector.

Please show me a caching front-end with a dynamic back-end that you control and that you think is protected against cache busting and we'll discuss that off-line. I'd hate to expose you to some joker taking you down just to make the point.

A cache can even become it's own enemy, for instance, if it passes requests on to the backend server if the GET contains url parameters and these make the request unique in the eyes of the cache but not unique in the eyes of the back-end then the cache could easily eat up a ton of memory even though from the back-ends perspective the same page got hit over and over again.

Caching is tricky.

That's what wildcards are for surely. We cover the same cases in MVC routers.

I moved my blog from Wordpress to a static site but I'm actually thinking of returning to Wordpress. Wordpress gives you comments, RSS feeds, search, categorisation and mailing list subscriptions out of the box. I'm missing those features in my static site. I don't want to outsource comments to Disqus. And there are caching techniques you can use to speed up Wordpress.

I'm guessing most of the static site generators probably fulfilled the needs of the programmer who wrote them (which is totally fine and kudos to them for releasing their work so others could benefit). However, static site generators are not what you'd call "user-friendly" (in my view) unless you're comfortable with the command line and like markdown. (Wordpress isn't particulalry user-friendly either, but more so than static site generators in my opinion.)

Static can give you RSS feeds, and most of the other bits & pieces can be outsourced these days to parties that specialize in exactly that.

I'm working on a new version of Gus[0] that will be able to figure out only what needs to be rebuilt and rebuild it.

Another feature for the (not to distant?) future will be generating a TF-IDF index in javascript so that a client-side search functionality could be introduced.

I also find Gus to have a pretty low bar to publishing. I write in markdown/textile/reStructuredText, build, and rsync. Granted I haven't used many other static generators (the ones I did look at didn't seem to have the features I wanted like tag or date indexes, plus it was one of my first python projects).

[0]: https://github.com/jimktrains/gus

> figure out only what needs to be rebuilt and rebuild it.

I thought a number of SSGs already did this.

They may. Since starting on Gus, I haven't really tried too many others:\

The OP complained about it and I was just pointing out that it doesn't have to be that all pages are recompiled.

CMS's won for a reason. I used to admin a Collage install, inherited from an acquisition. What a piece of shit that experience was for all involved. On top of it, everything dynamic had to be tacked on in a 90's web style with certain pages being dynamic, a bit for discussion, a different bit for login, a different bit for whatever, etc. A CMS is at least one system and these things are treated as plugins as opposed to 20 different vendors and whatever hackey scripting to keep everything talking to everything else. Even end users could see how rickety the whole production was. There's something to be said about a central monolithic structure with plugins vs a collection of scripts and html.

Deployment was usually frustrating and broke in non-obvious ways. Deploying a non-trivial Collage site was like gambling.

The experience of switching to Drupal was like night and day. Any bullshit concerns about performance were taken care of with just Varnish or a caching plugin. I can't imagine going back to an SSG and an ecosystem of disparate webapps to make it a usable website. Heck, even my side projects with Wordpress are like switching from a Model T to a Tesla compared to dealing with an SSG. Maybe SSG's are fun for sites with low requirements, but once you have a non-trivial set of features to support, then you're doing extra work and experiencing extra pain that a CMS wouldn't give you.

Also your coworkers wont be cursing your name after you leave them a patchwork of barely held together code in your wake to make the SSG based site do dynamic things CMS's find trivial. Especially if they come from a CMS background.

"Also your coworkers wont be cursing your name after you leave them a patchwork of barely held together code in your wake to make the SSG based site do dynamic things CMS's find trivial. Especially if they come from a CMS background."

Can you provide an example of something that CMS's find trivial that is difficult to do in an SSG?

> "live pages" equals "lots of hits to the database"

Is it common for people to set up a database backed website without caching? The framework I'm most familiar with, Django, will cache the results of database queries, so if, say, an article on the site becomes popular the database is hit once, and thousands of views are served from the cache.

Edit: Jweb_Guru, thanks for that clarification.

That isn't true by default. Django will cache queries the first time they're executed for a particular request, but not in general for all requests. You can opt-in to caching pages thereafter in something like memcached, but that would be extremely questionable default behavior and a great way to cause consistency bugs.

Wordpress uses plugins for caching, last I checked.

Incremental compilation has been around for decades. I have a couple of scripts that generate static HTML pages for parts of my website, and I use *make -- and when I make a single change, only its corresponding page is updated. SSGs don't ever need to rebuild from scratch. Just leverage existing make/ninja/etc tools for your blog...

I agree with your assessment. To me the reason we keep ping-pnging between the two extremes is a lack of standardized and common caching mechanism. In a perfect world every web server would be behind an HTTP reverse proxy and we could just use HTTP headers to control caching. But there are a lot of quirks to that so unless you control the whole stack you don't have a proxy out front. And then you get into all other kinds of caching schemes: filesystem, memcache, redis, s3/cloudfront, etc. Which one you choose is determined by your tech stack, your PaaS, etc. Which the CMS creators have limited control over. If caching was a no brainer CMS creators could just concentrate on making the dynamic part good.

Agreed. Essentially SSGs are caching - or rather they are a way of pushing the caching problem down onto a lower level.

It's no different to 'clear the whole cache every time I change any content and store it all in memcached' - you're just substituting 'file-system' for 'memcached'.

Of course - SSG do solve another problem - that of needing complex software running on your server - you can run SSGs locally or on a centralized server - but that's not what is being discussed here.

The crucial difference, as others have pointed out, is that if you start with a dynamic site and add caching, there are a lot of likely gotchas like infinite url's that all load a slightly different page (pagination, tag combinations, etc.), as well as potential security issues when a page is not cached yet, or the cache is busted.

On the other hand, if you start with a SSG, your initial state is that every page is cached and works, and you have to think carefully of adding some kind of pagination mechanism.

As the quote goes, "There are only two hard problems in Computer Science: cache invalidation and naming things." The big benefit of a SSG is that you 'solved' the cache problem, and will immediately be aware of issues there.

> It's no different to 'clear the whole cache every time I change any content and store it all in memcached' - you're just substituting 'file-system' for 'memcached'

There's no reason why the SSG needs to rebuild the entire site if you've only changed one thing.

True but my point still stands - some caches are dumb and some are smart. Some SSGs rebuild everything and some can figure out what needs to be rebuilt.

I agree with what you are saying, and I think this is perhaps the most important tradeoff that many CMS designs do not deal with properly.

This is particularly true of the current plethora of 'static site generators', many of which are not really improvements in any aspect.

If you build a static site, you can concentrate on making sure a rebuild is quick; if you build a dynamic site, you can concentrate on making sure the data structure is robust to scaling.

You might never be without these problems, but you can minimise them to the extent that they are no longer costing your users excessively.

It's not that we need to go back and forth: either solution could work perfectly fine for all but the heaviest sites.

Not necessarily so. More and more, we're seeing rich-content in-browser website builders which allow customers to have their lunch and eat it. You get to customize and modify your page (optionally with WYSIWYG) online, publish it on the fly and have it generated and stored in cache somewhere in the blink of an eye. Just today, two web applications doing just that have hit the home page of HN.

It doesn't have to be either/or and I don't think the market really "oscillates" in that regard, in my opinion it will eventually provide users the best of both worlds (and maybe already does), at least for more or less static websites.

> it requires a "compilation"/"rebuilding" step that gets longer and longer as your site gets bigger

Unless the generator can keep track of what has and hasn't changed and only rebuild the changed stuff.

Obvious gotcha to that is if you have a link on every blog post to jump to the most recent entry. If you were going dynamic, that info would be inserted on page generation, if you're going static then you need to update everything.

Easy solution would be to make a duplicate page like /latest.html which updates as necessary.

There are other gotchas as well; for example, on my blog I have links in my sidebar to pages for categories and tags, but the links have total post counts for the category/tag after them, so every time a new post is added at least one count will change on every page. I don't mind because the SSG I use (my own, https://github.com/pdonis/simpleblog3 ) renders fast enough to suit me; it takes about 5-6 seconds to render my entire blog.

Since the sidebar stuff or 'recent entry' links are not really important for SEO, couldn't you just load those from one static file using a bit of javascript? That's what I'm thinking of doing...

I could, but I want to keep the site entirely Javascript-free. I have experimented a bit with using iframes to load the sidebars from one static file, but I haven't gotten that to work the way I want it to.

Ah, then a javascript solution is out of the question. What problem did you run into with iframes?

Browsers don't seem to treat iframes the way they treat other layout elements; I have to specify the exact size (width and height) of the iframe or it won't get laid out correctly. I don't want that; I want an iframe that flows with the rest of the layout just like other box elements.

You can get the best of both by using a dynamic site CMS, and then hooking it up to Amazon Cloudfront to cache your html. The only challenge then is page cache expiration. You can set that by either using the http headers for Cache-Control or setting page rules in Cloudfront's admin.

Such a setup will scale like crazy and skip "compilation" steps

The challenge isn't page caching but object caching and rendering.

SSGs are just a rerun of the 90's and don't solve anything that Varnish doesn't already. Pages are an increasingly quaint notion on an increasingly dynamic, personalised, multi-device web. Time to publish is ever more important as Google SEO measures news sources in minutes and publishing workflows often require instant publishing and internal search where there are handoffs (e.g. a subeditor and picture desk).

The two real problems I see is that ESI is our best solution to rendering (and is a clunky 90s-style design), and that object cache refresh has a tendency to combinatorial explosion.

While it may be shiny and hip to provide the latest and greatest news with very fast publishing cycles, I am not sure if this is for everyone.

Thoughtful articles require time to write and edit, reviewers will spend time on them. Re-running a static site generator to publish them is a fraction of the total time it requires to write a good article.

If, on the other hand, you are not in for quality, then yes, time to publish may be important for you.

That's really just the difference between news and articles. News events (big trials, disasters etc) are highly time-sensitive and if your income depends on display ads and Google, instant publishing is crucial and something that comes up on RFPs for such systems. Workflow where content is created by multiple people isn't as well understood but can be a source of incredible friction and productivity killer in publishing.

The argument is sometimes made that this doesn't apply to simple blogging and that SSGs of course aren't aimed at pros, but I can say with some certainty that the segment of bloggers who can and want to publish with Git etc is infinitesimally small and quite possibly 100% represented on HN :) That's fine and great if it works for your use case but it pains me a bit to see people re-discovering the very old and disproven model that SSG represents. It's a) highly niche and b) counter to many current and new requirements in content publishing.

The main argument for it is performance but I don't see how it improves on Varnish - albeit w cache warming - and tellingly edge cases of cache warming are getting replayed in SSG increasing complexity. In fact dynamic systems often forgo cache warming as not worth the result much of the time (and can do it selectively), but you don't get that control at all in an SSG system.

This really hits home for me, because I'd love to use a CMS, but I just haven't found that is: (a) easy to use (b) easy to customize.

Just to single out a shortcomings I've recently found: multiple authors with multiple languages, defining short and long version of bio/tagline. The tagline is displayed at the article, which may be tagged with multiple contributors with different roles (author, translator, reviewer), while the author page displays the long version.

I've spent days to figure out what plugin/theme combination (Wordpress or Drupal) would enable me this single feature, and I couldn't figure it out.

Yesterday evening I gave up, and now I have <200 lines of code in Dart (using markdown and mustache), generating static pages that were annotated with metadata, doing just that. To be honest, I'd trade it any time for a decent CMS, I just can't find any.

No, the main argument is for security, then cost (software updates cost time), then perhaps performance, though as you point out there isn't much difference with a properly configured cache.

Clearly static doesn't suit all sites but if the site is infrequently changed and not too complex or dynamic it is sometimes the best choice. If you have highly dynamic content with multiple authors static is clearly a terrible choice. This is not a zero sum game, there is room for both approaches.

Given even the White House uses a db backed CMS, security seems a pretty weak argument to be making. Yes, static HTML is sometimes an excellent choice but it doesn't represent any kind of new solution in the space.

Speaking of swinging pendulums in content management systems and wrong things, don't you ever get sick of http(s)? Http gets it wrong in many ways too, but I don't hear much about alternatives.