WordPress To Jekyll And Back Again | Hacker News Reader

WordPress To Jekyll And Back Again

brandontreb.com

48 pointsbrandontreb43 comments

I can't tell from the site itself, since it's non-responsive (back to WordPress?), but from the comments here it appears the author objects to Jekyll regenerating the entire site every time.

If that's the case then I suggest he tries Hakyll, which has a dependency system built in so that it only rebuilds what it needs when files change.

http://jaspervdj.be/hakyll/

haha ironic that it's crashing now eh? That's the one upside of Jekyll for sure. I was hosting the site on 512MB of ram. Just did a resize to handle the HN traffic, hopefully it comes back online soon.

Hakyll looks pretty slick. I'll check it out for sure. Thanks

Wow, you left this comment two hours ago, and the site is still down. I see you are hosting on Rackspace. Something going wrong?

Rackspace. It's taking forever to resize. Site isn't _quite_ down. Just super slow.

Have you implemented any of the caching tools for WordPress? I also run WordPress on my blog, which is hosted at Dreamhost (shared hosting), which is about the worst you can have. I've never been hit terribly hard, but I did manage to pick up around 2,000 visitors in the span of about 3 hours one time, which is the most my blog has ever seen. WP with WP SuperCache kept it up, even on a shared host.

I'd be curious to read a post-mortem once you get the site back up.

I had my wordpress blog on shared Dreamhost for about a year.. I was slowly increasing traffic then I noticed the dreamhost daemons killing off my processes.. after much, much hassle I finally got the offical line of "1000 unique visitors per day is using too much memory for Dreamhost shared - you need to upgrade". I moved to rackspace.

I'm betting on no.

Caching with WordPress.org is as easy to set up as Jekyll, if not easier. With W3 Total Cache or WP Super Cache, performance is roughly equivalent to a baked site being hosted by the same web server.

Edit: Oops, I lost, as he already said in another thread that he had WP Super Cache turned on. But it seems to me that 512MB would be enough if the web server was set up properly, and in fact caching was set up so the web server would serve pages directly.

Maybe I didn't get something about this static blog generating trend thing, but what would be wrong about having a 'normal' database-backed blog software that simply renders the pages staticly to a file (as it is an option with Rails caching) ? That would work with some simple inexpensive DB like SQLite, but still leave the system flexible and easy to edit with a backend.

http://www.idlewords.com/2009/09/using_wordpress_to_generate...

That's how movable type, the predecessor to wordpress works. It's written in perl.

There's a popular Wordpress plugin which does something similar called WP Super cache.

http://wordpress.org/extend/plugins/wp-super-cache/

I use wp-super-cache for this. However, it's still not the same as static site generation. The entire Wordpress system still gets loaded at least to the plugin loading phase.

There are numerous db hits that take place before this stage (mainly to figure out which plugins need to be loaded).

Also, the PHP must be interpreted in order to render html. The static site generation has _none_ of this as it's already pre-rendered in html.

OK, so obviously I suck at server configuration/optimization. Any tips for optimizations that could have enabled me to support the influx of Hacker News traffic?

Note:

* Ubuntu 512MB on Rackspace

* Apache

* Wordpress with wp-super-cache and minimal plugins enabled

Thanks for the feedback!

* Ditching Apache for nginx will make the biggest impact.

* Run PHP with php-fpm (bundled in latest PHP 5.3)

* Use a PHP accelerator like APC.

Awesome advice. Thanks!

I had a similar issue a while back after having a blog post of mine featured on HN front page for some time. The problem is really apache. Either configure apache with workers (which is ain't the default config) or add nginx in front. You might be interested in this particular post:

http://blog.rassemblr.com/2011/01/wordpress-need-for-speed-o...

This has been throughly tested with yet another post on HN front page a while later! My blog handled the load without a glitch.

Looks like I'm replying with the same comment over and over again. But I have to share nginx love :-)

Regarding the time it takes to build the site - is everything rebuilt every time in Jekyll?

You'd think there would be some sort of dependency build tree ala make/rake so only items whose source components would need to be rebuilt (which, in some cases could be the whole site).

It has to regenerate the entire site each time. I guess the reason for this might be because it can't be sure every page is updated otherwise. Take for example adding a new post and you have a "recent posts" widget in your sidebar. This affects every page on the site and therefore it must be rebuilt.

Recent posts should just be loaded dynamically via javascript, the same way recent tweets are loaded.

Is parsing a page template on the server and including a 'recent posts' snippet really so challenging that it justifies every client doing another HTTP request for that very same snippet?

Dependencies are hard to figure out automatically and a bit of a nuisance if you'd have to declare whether every plugin/script use has dynamic side effects. Considering that jekyll is a pretty small (and somewhat opinionated) system by default, it probably wasn't considered worth the effort.

And once you approach the level of a more typical WordPress site in both volume and information per page, I'm not sure how much effort you'd save. It's probably better to investigate how to make the general run time a bit faster. (Although it would probably be possible to have mostly static pages with few dependencies, then make the dynamic/dependent changes via a minimal web service and AJAX. At least when the load is too much for that, you'd still see the static content. Wouldn't be as generic/small as jekyll then, though.)

I am amused by the site being down. Going back to WordPress, he presumably forgot that a cache plugin is pretty much required. Not that Jekyll is perfect, but that's one area where it's pretty much impossible to beat.

I actually have wp-supercache enabled. However, the site was only running on 512MB ram :( Getting featured on HN is a lot different than daily traffic. A static site would not have saved me here.

A static site would have saved you here, particularly if you are using something like nginx. Even with apache, you'd still be fine.

Wordpress cacheing HN traffic should do fine too. There's something else going on here.

Indeed. I've been on the HN front page a couple of times with Jekyll generated sites (http://os-blog.com is one). Hosted on an Amazon micro instance (613 mb) and served by nginx, none of my sites have broken a sweat under the increased traffic.

HN traffic isn't really that much. I was watching the stats under the new Google Analytics realtime mode and the site was averaging about 3 - 6 requests per second.

One more "geekCred--;" for letting WordPress (or whatever) convert two '-' to a single '–'.

More on topic, though: I don't think anyone is seriously claiming that both solutions are equivalent or easily interchangeable. For me the simple and 'process the files once' approach makes a lot of sense and even seems more transparent than a big beast like WordPress. Whether Jekyll or Blosxom - something that feels less like a CMS and more like a way to write simple texts.

1: http://www.blosxom.com/

haha I'll definitely have to fix that :) Blosxom looks interesting, does it support markdown?

Grubers original Markdown implementation[1] comes with a blosxom plugin.

[1]: http://daringfireball.net/projects/markdown/

Regarding your use of Markdown, I had a similar experience.

I've had Markdown on my WordPress install for many years, but only recently decided to commit to learning the main commands off by heart, and use it whenever possible. Markdown's not perfect, but committing to it has sped up writing a lot.

There are some solid Markdown graphical text editors too, if you're into that sort of thing. Most of these allow you to preview your documents while writing them, which is a big plus.

I developed MarkdownPad (http://markdownpad.com) for Windows, and there's also Mou for Mac and Retext for Linux.

You could also Blog like a Hacker[1], and use no servers at all, like Amazon's CTO Werner Vongels[2].

You can see more from his own comments when he posted it on Hacker news[3].

[1] http://tom.preston-werner.com/2008/11/17/blogging-like-a-hac...

[2] http://www.allthingsdistributed.com/2011/08/Jekyll-amazon-s3...

[3] http://news.ycombinator.com/item?id=2896860

you can also do this with WordPress http://wpengine.com/faq/blog-s3/

(you can run WordPress locally to generate the site, or use a 3rd party service just to run the process but not serve from it).

Except that this is not a hacker solution, as you need to actually go to http://wpengine.com/contact/ , fill a form, and wait.

The hacker solution that the Chief Technology Officer of Amazon uses is much more a "power to the people" approach.

No, as I wrote, you can run WordPress locally (ie on your own workstation, local server, whatever) and have it send to S3.

I gave the WPEngine link just to prove people are doing it with WordPress and it's not a pipedream.

Then the time required for a complete generation is just the same.

The author obviously don't want to compile the site as it takes too long for him, so compiling a wordpress blog would not yield any time benefits.

I like that I can run my Jekyll blog for free on Heroku, and it is served entirely out of Varnish. Seems like you are paying more money for crap performance.

In addition, my blog regenerates in < 5 seconds on Ruby 1.9.3.

A week ago, Jack Baty, also having been allured by static site generation hotness, switched back to Wordpress after it took 8 minutes to regenerate his Jekyll blog with over 1,600 posts. "Life's too short."

http://jackbaty.com/2011/10/new-blogging-hotness/

At the moment, Jekyll (via Octopress) is more interesting/educational to me than plugging away at familiar Wordpress, so I'll cross that bridge when I get there.

I remember when everybody moved on to WordPress while I stayed on MovableType 2.6. I finally moved one year and a half later, when my 400+ posts made the building time absolutely unbearable.

Are there any Jekyll blogs with hundreds or thousands of posts?

Paul Stamatiou switched to Jekyll a while back and has hundreds of posts, I believe.

http://paulstamatiou.com

I am about to move my personal blog over to blogofile (like Jekyll, but written in Python) which has about 500 or so entries. Will let you know how long they take to build when done...

If the runtime was the main issue, couldn't you just mod jekyll to only regen based on new data (modding site.process basically)...obviously if you changed templating information you'd want to regen everything, but....

Wish i could read the blog post. I am quite a happy Jekyll user.

And while Jekyll still has some downsides it is superior when it comes to handling a load spike from a HN post.

Still can't load the website. If anyone could paste the text of the article here. :-) (WordPress To Jekyll And Back Again And Finally To HN)

Reply on news.ycombinator.com